2023-02-13 12:15:27,092 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-13 12:15:27,334 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa 2023-02-13 12:15:27,351 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 12:15:27,352 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa, deleteOnExit=true 2023-02-13 12:15:27,352 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 12:15:27,353 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/test.cache.data in system properties and HBase conf 2023-02-13 12:15:27,353 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 12:15:27,354 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/hadoop.log.dir in system properties and HBase conf 2023-02-13 12:15:27,355 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 12:15:27,355 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 12:15:27,355 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 12:15:27,456 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 12:15:27,832 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 12:15:27,838 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 12:15:27,839 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 12:15:27,839 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 12:15:27,839 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 12:15:27,840 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 12:15:27,840 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 12:15:27,841 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 12:15:27,841 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 12:15:27,842 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 12:15:27,842 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/nfs.dump.dir in system properties and HBase conf 2023-02-13 12:15:27,843 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/java.io.tmpdir in system properties and HBase conf 2023-02-13 12:15:27,843 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 12:15:27,843 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 12:15:27,844 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 12:15:28,358 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 12:15:28,363 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 12:15:29,327 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 12:15:29,514 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 12:15:29,528 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 12:15:29,561 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 12:15:29,587 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/java.io.tmpdir/Jetty_localhost_localdomain_36263_hdfs____.xriinu/webapp 2023-02-13 12:15:29,726 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:36263 2023-02-13 12:15:29,735 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 12:15:29,736 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 12:15:30,218 WARN [Listener at localhost.localdomain/44273] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 12:15:30,302 WARN [Listener at localhost.localdomain/44273] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 12:15:30,319 WARN [Listener at localhost.localdomain/44273] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 12:15:30,325 INFO [Listener at localhost.localdomain/44273] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 12:15:30,329 INFO [Listener at localhost.localdomain/44273] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/java.io.tmpdir/Jetty_localhost_39571_datanode____.ny7a40/webapp 2023-02-13 12:15:30,411 INFO [Listener at localhost.localdomain/44273] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39571 2023-02-13 12:15:30,670 WARN [Listener at localhost.localdomain/45625] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 12:15:30,687 WARN [Listener at localhost.localdomain/45625] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 12:15:30,692 WARN [Listener at localhost.localdomain/45625] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 12:15:30,693 INFO [Listener at localhost.localdomain/45625] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 12:15:30,700 INFO [Listener at localhost.localdomain/45625] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/java.io.tmpdir/Jetty_localhost_40839_datanode____1s3ybt/webapp 2023-02-13 12:15:30,778 INFO [Listener at localhost.localdomain/45625] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40839 2023-02-13 12:15:30,788 WARN [Listener at localhost.localdomain/37021] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 12:15:30,807 WARN [Listener at localhost.localdomain/37021] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 12:15:30,812 WARN [Listener at localhost.localdomain/37021] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 12:15:30,814 INFO [Listener at localhost.localdomain/37021] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 12:15:30,818 INFO [Listener at localhost.localdomain/37021] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/java.io.tmpdir/Jetty_localhost_40237_datanode____3ycy69/webapp 2023-02-13 12:15:30,895 INFO [Listener at localhost.localdomain/37021] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40237 2023-02-13 12:15:30,906 WARN [Listener at localhost.localdomain/46087] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 12:15:31,755 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfd135fc44942cdb4: Processing first storage report for DS-76042f15-cd3b-4856-a50f-9b89070b594e from datanode 32c2d397-91bb-4523-b299-5dcd9d7d748d 2023-02-13 12:15:31,757 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfd135fc44942cdb4: from storage DS-76042f15-cd3b-4856-a50f-9b89070b594e node DatanodeRegistration(127.0.0.1:33121, datanodeUuid=32c2d397-91bb-4523-b299-5dcd9d7d748d, infoPort=34239, infoSecurePort=0, ipcPort=37021, storageInfo=lv=-57;cid=testClusterID;nsid=1073841053;c=1676290528441), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-13 12:15:31,757 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x519795baec954549: Processing first storage report for DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f from datanode 14b0d84a-bcac-40f1-8e55-f251954dc488 2023-02-13 12:15:31,757 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x519795baec954549: from storage DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f node DatanodeRegistration(127.0.0.1:34415, datanodeUuid=14b0d84a-bcac-40f1-8e55-f251954dc488, infoPort=42281, infoSecurePort=0, ipcPort=45625, storageInfo=lv=-57;cid=testClusterID;nsid=1073841053;c=1676290528441), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 12:15:31,758 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x28419f5f11b1f469: Processing first storage report for DS-16a8c210-c13d-4264-abac-6466dde30377 from datanode c0f6d6b4-56b4-4c38-a07d-ddec5113cb94 2023-02-13 12:15:31,758 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x28419f5f11b1f469: from storage DS-16a8c210-c13d-4264-abac-6466dde30377 node DatanodeRegistration(127.0.0.1:38573, datanodeUuid=c0f6d6b4-56b4-4c38-a07d-ddec5113cb94, infoPort=33581, infoSecurePort=0, ipcPort=46087, storageInfo=lv=-57;cid=testClusterID;nsid=1073841053;c=1676290528441), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 12:15:31,758 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfd135fc44942cdb4: Processing first storage report for DS-cec574a6-0b7d-4273-87da-c55dd7c99c83 from datanode 32c2d397-91bb-4523-b299-5dcd9d7d748d 2023-02-13 12:15:31,758 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfd135fc44942cdb4: from storage DS-cec574a6-0b7d-4273-87da-c55dd7c99c83 node DatanodeRegistration(127.0.0.1:33121, datanodeUuid=32c2d397-91bb-4523-b299-5dcd9d7d748d, infoPort=34239, infoSecurePort=0, ipcPort=37021, storageInfo=lv=-57;cid=testClusterID;nsid=1073841053;c=1676290528441), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 12:15:31,758 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x519795baec954549: Processing first storage report for DS-36e4af53-62b6-479a-a15c-4eb05f21c966 from datanode 14b0d84a-bcac-40f1-8e55-f251954dc488 2023-02-13 12:15:31,758 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x519795baec954549: from storage DS-36e4af53-62b6-479a-a15c-4eb05f21c966 node DatanodeRegistration(127.0.0.1:34415, datanodeUuid=14b0d84a-bcac-40f1-8e55-f251954dc488, infoPort=42281, infoSecurePort=0, ipcPort=45625, storageInfo=lv=-57;cid=testClusterID;nsid=1073841053;c=1676290528441), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 12:15:31,758 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x28419f5f11b1f469: Processing first storage report for DS-da23820e-82d5-4157-94f0-e59c2183620a from datanode c0f6d6b4-56b4-4c38-a07d-ddec5113cb94 2023-02-13 12:15:31,759 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x28419f5f11b1f469: from storage DS-da23820e-82d5-4157-94f0-e59c2183620a node DatanodeRegistration(127.0.0.1:38573, datanodeUuid=c0f6d6b4-56b4-4c38-a07d-ddec5113cb94, infoPort=33581, infoSecurePort=0, ipcPort=46087, storageInfo=lv=-57;cid=testClusterID;nsid=1073841053;c=1676290528441), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 12:15:31,834 DEBUG [Listener at localhost.localdomain/46087] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa 2023-02-13 12:15:31,884 INFO [Listener at localhost.localdomain/46087] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/zookeeper_0, clientPort=54988, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 12:15:31,895 INFO [Listener at localhost.localdomain/46087] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=54988 2023-02-13 12:15:31,901 INFO [Listener at localhost.localdomain/46087] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:31,903 INFO [Listener at localhost.localdomain/46087] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:32,539 INFO [Listener at localhost.localdomain/46087] util.FSUtils(479): Created version file at hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8 with version=8 2023-02-13 12:15:32,539 INFO [Listener at localhost.localdomain/46087] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/hbase-staging 2023-02-13 12:15:32,794 INFO [Listener at localhost.localdomain/46087] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 12:15:33,145 INFO [Listener at localhost.localdomain/46087] client.ConnectionUtils(127): master/jenkins-hbase16:0 server-side Connection retries=45 2023-02-13 12:15:33,170 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,170 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,171 INFO [Listener at localhost.localdomain/46087] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 12:15:33,171 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,171 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 12:15:33,296 INFO [Listener at localhost.localdomain/46087] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 12:15:33,351 DEBUG [Listener at localhost.localdomain/46087] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 12:15:33,425 INFO [Listener at localhost.localdomain/46087] ipc.NettyRpcServer(120): Bind to /188.40.62.62:39699 2023-02-13 12:15:33,434 INFO [Listener at localhost.localdomain/46087] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:33,436 INFO [Listener at localhost.localdomain/46087] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:33,454 INFO [Listener at localhost.localdomain/46087] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39699 connecting to ZooKeeper ensemble=127.0.0.1:54988 2023-02-13 12:15:33,550 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:396990x0, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 12:15:33,553 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:39699-0x10052d110940000 connected 2023-02-13 12:15:33,628 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 12:15:33,630 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:15:33,637 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 12:15:33,646 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39699 2023-02-13 12:15:33,647 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39699 2023-02-13 12:15:33,647 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39699 2023-02-13 12:15:33,648 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39699 2023-02-13 12:15:33,648 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39699 2023-02-13 12:15:33,653 INFO [Listener at localhost.localdomain/46087] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8, hbase.cluster.distributed=false 2023-02-13 12:15:33,709 INFO [Listener at localhost.localdomain/46087] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=45 2023-02-13 12:15:33,709 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,710 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,710 INFO [Listener at localhost.localdomain/46087] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 12:15:33,710 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,710 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 12:15:33,714 INFO [Listener at localhost.localdomain/46087] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 12:15:33,717 INFO [Listener at localhost.localdomain/46087] ipc.NettyRpcServer(120): Bind to /188.40.62.62:33055 2023-02-13 12:15:33,719 INFO [Listener at localhost.localdomain/46087] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 12:15:33,724 DEBUG [Listener at localhost.localdomain/46087] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 12:15:33,725 INFO [Listener at localhost.localdomain/46087] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:33,727 INFO [Listener at localhost.localdomain/46087] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:33,728 INFO [Listener at localhost.localdomain/46087] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33055 connecting to ZooKeeper ensemble=127.0.0.1:54988 2023-02-13 12:15:33,740 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:330550x0, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 12:15:33,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33055-0x10052d110940001 connected 2023-02-13 12:15:33,741 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 12:15:33,743 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:15:33,744 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 12:15:33,745 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33055 2023-02-13 12:15:33,745 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33055 2023-02-13 12:15:33,745 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33055 2023-02-13 12:15:33,746 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33055 2023-02-13 12:15:33,746 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33055 2023-02-13 12:15:33,759 INFO [Listener at localhost.localdomain/46087] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=45 2023-02-13 12:15:33,760 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,760 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,760 INFO [Listener at localhost.localdomain/46087] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 12:15:33,760 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,760 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 12:15:33,761 INFO [Listener at localhost.localdomain/46087] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 12:15:33,763 INFO [Listener at localhost.localdomain/46087] ipc.NettyRpcServer(120): Bind to /188.40.62.62:33961 2023-02-13 12:15:33,763 INFO [Listener at localhost.localdomain/46087] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 12:15:33,764 DEBUG [Listener at localhost.localdomain/46087] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 12:15:33,765 INFO [Listener at localhost.localdomain/46087] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:33,767 INFO [Listener at localhost.localdomain/46087] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:33,768 INFO [Listener at localhost.localdomain/46087] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33961 connecting to ZooKeeper ensemble=127.0.0.1:54988 2023-02-13 12:15:33,782 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:339610x0, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 12:15:33,783 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33961-0x10052d110940002 connected 2023-02-13 12:15:33,783 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 12:15:33,784 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:15:33,785 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 12:15:33,785 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33961 2023-02-13 12:15:33,786 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33961 2023-02-13 12:15:33,786 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33961 2023-02-13 12:15:33,786 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33961 2023-02-13 12:15:33,786 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33961 2023-02-13 12:15:33,800 INFO [Listener at localhost.localdomain/46087] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=45 2023-02-13 12:15:33,800 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,800 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,800 INFO [Listener at localhost.localdomain/46087] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 12:15:33,800 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:15:33,801 INFO [Listener at localhost.localdomain/46087] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 12:15:33,801 INFO [Listener at localhost.localdomain/46087] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 12:15:33,802 INFO [Listener at localhost.localdomain/46087] ipc.NettyRpcServer(120): Bind to /188.40.62.62:45345 2023-02-13 12:15:33,802 INFO [Listener at localhost.localdomain/46087] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 12:15:33,803 DEBUG [Listener at localhost.localdomain/46087] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 12:15:33,804 INFO [Listener at localhost.localdomain/46087] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:33,806 INFO [Listener at localhost.localdomain/46087] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:33,807 INFO [Listener at localhost.localdomain/46087] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45345 connecting to ZooKeeper ensemble=127.0.0.1:54988 2023-02-13 12:15:33,818 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:453450x0, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 12:15:33,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:45345-0x10052d110940003 connected 2023-02-13 12:15:33,819 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 12:15:33,820 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:15:33,821 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 12:15:33,821 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45345 2023-02-13 12:15:33,821 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45345 2023-02-13 12:15:33,822 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45345 2023-02-13 12:15:33,822 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45345 2023-02-13 12:15:33,822 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45345 2023-02-13 12:15:33,824 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase16.apache.org,39699,1676290532646 2023-02-13 12:15:33,840 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 12:15:33,842 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase16.apache.org,39699,1676290532646 2023-02-13 12:15:33,867 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 12:15:33,868 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 12:15:33,867 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 12:15:33,867 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 12:15:33,868 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:15:33,870 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 12:15:33,871 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase16.apache.org,39699,1676290532646 from backup master directory 2023-02-13 12:15:33,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 12:15:33,884 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase16.apache.org,39699,1676290532646 2023-02-13 12:15:33,885 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 12:15:33,885 WARN [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 12:15:33,886 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase16.apache.org,39699,1676290532646 2023-02-13 12:15:33,889 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 12:15:33,891 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 12:15:33,979 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/hbase.id with ID: 3bfa7887-571a-43a3-aa8a-ab0d4e6ebb73 2023-02-13 12:15:34,019 INFO [master/jenkins-hbase16:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:15:34,042 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:15:34,092 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x55d858b9 to 127.0.0.1:54988 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:15:34,134 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c873dfb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:15:34,153 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 12:15:34,154 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 12:15:34,168 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 12:15:34,168 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 12:15:34,170 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 12:15:34,173 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 12:15:34,174 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 12:15:34,201 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/data/master/store-tmp 2023-02-13 12:15:34,235 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:34,236 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 12:15:34,236 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:15:34,236 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:15:34,236 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 12:15:34,236 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:15:34,236 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:15:34,237 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676290534235Disabling compacts and flushes for region at 1676290534236 (+1 ms)Disabling writes for close at 1676290534236Writing region close event to WAL at 1676290534236Closed at 1676290534236 2023-02-13 12:15:34,238 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/WALs/jenkins-hbase16.apache.org,39699,1676290532646 2023-02-13 12:15:34,256 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C39699%2C1676290532646, suffix=, logDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/WALs/jenkins-hbase16.apache.org,39699,1676290532646, archiveDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/oldWALs, maxLogs=10 2023-02-13 12:15:34,305 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34415,DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f,DISK] 2023-02-13 12:15:34,305 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38573,DS-16a8c210-c13d-4264-abac-6466dde30377,DISK] 2023-02-13 12:15:34,305 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33121,DS-76042f15-cd3b-4856-a50f-9b89070b594e,DISK] 2023-02-13 12:15:34,312 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 12:15:34,372 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/WALs/jenkins-hbase16.apache.org,39699,1676290532646/jenkins-hbase16.apache.org%2C39699%2C1676290532646.1676290534265 2023-02-13 12:15:34,373 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38573,DS-16a8c210-c13d-4264-abac-6466dde30377,DISK], DatanodeInfoWithStorage[127.0.0.1:34415,DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f,DISK], DatanodeInfoWithStorage[127.0.0.1:33121,DS-76042f15-cd3b-4856-a50f-9b89070b594e,DISK]] 2023-02-13 12:15:34,374 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:15:34,374 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:34,377 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:15:34,378 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:15:34,428 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:15:34,434 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 12:15:34,455 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 12:15:34,466 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:34,471 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:15:34,473 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:15:34,488 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:15:34,493 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:15:34,494 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68799116, jitterRate=0.02518671751022339}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:15:34,494 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676290534400Initializing all the Stores at 1676290534402 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290534403 (+1 ms)Cleaning up temporary data from old regions at 1676290534479 (+76 ms)Cleaning up detritus from prior splits at 1676290534480 (+1 ms)Region opened successfully at 1676290534494 (+14 ms) 2023-02-13 12:15:34,495 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 12:15:34,512 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:34,512 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:34,514 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:34,516 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:34,541 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 25 msec 2023-02-13 12:15:34,541 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 12:15:34,562 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 12:15:34,567 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 12:15:34,587 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 12:15:34,590 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 12:15:34,591 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 12:15:34,595 INFO [master/jenkins-hbase16:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 12:15:34,599 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 12:15:34,634 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:15:34,636 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 12:15:34,637 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 12:15:34,648 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 12:15:34,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 12:15:34,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 12:15:34,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 12:15:34,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 12:15:34,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:15:34,660 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase16.apache.org,39699,1676290532646, sessionid=0x10052d110940000, setting cluster-up flag (Was=false) 2023-02-13 12:15:34,690 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:15:34,718 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 12:15:34,720 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,39699,1676290532646 2023-02-13 12:15:34,742 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:15:34,767 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 12:15:34,769 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,39699,1676290532646 2023-02-13 12:15:34,771 WARN [master/jenkins-hbase16:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.hbase-snapshot/.tmp 2023-02-13 12:15:34,826 INFO [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(952): ClusterId : 3bfa7887-571a-43a3-aa8a-ab0d4e6ebb73 2023-02-13 12:15:34,826 INFO [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(952): ClusterId : 3bfa7887-571a-43a3-aa8a-ab0d4e6ebb73 2023-02-13 12:15:34,826 INFO [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(952): ClusterId : 3bfa7887-571a-43a3-aa8a-ab0d4e6ebb73 2023-02-13 12:15:34,830 DEBUG [RS:0;jenkins-hbase16:33055] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 12:15:34,830 DEBUG [RS:1;jenkins-hbase16:33961] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 12:15:34,830 DEBUG [RS:2;jenkins-hbase16:45345] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 12:15:34,852 DEBUG [RS:0;jenkins-hbase16:33055] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 12:15:34,853 DEBUG [RS:0;jenkins-hbase16:33055] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 12:15:34,852 DEBUG [RS:2;jenkins-hbase16:45345] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 12:15:34,852 DEBUG [RS:1;jenkins-hbase16:33961] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 12:15:34,853 DEBUG [RS:2;jenkins-hbase16:45345] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 12:15:34,853 DEBUG [RS:1;jenkins-hbase16:33961] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 12:15:34,869 DEBUG [RS:1;jenkins-hbase16:33961] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 12:15:34,869 DEBUG [RS:2;jenkins-hbase16:45345] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 12:15:34,869 DEBUG [RS:0;jenkins-hbase16:33055] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 12:15:34,871 DEBUG [RS:1;jenkins-hbase16:33961] zookeeper.ReadOnlyZKClient(139): Connect 0x3155f0d9 to 127.0.0.1:54988 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:15:34,871 DEBUG [RS:2;jenkins-hbase16:45345] zookeeper.ReadOnlyZKClient(139): Connect 0x3ce9878e to 127.0.0.1:54988 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:15:34,871 DEBUG [RS:0;jenkins-hbase16:33055] zookeeper.ReadOnlyZKClient(139): Connect 0x4a8de98e to 127.0.0.1:54988 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:15:34,872 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:34,901 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 12:15:34,901 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 12:15:34,901 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 12:15:34,901 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 12:15:34,902 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase16:0, corePoolSize=10, maxPoolSize=10 2023-02-13 12:15:34,902 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:34,902 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-02-13 12:15:34,902 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:34,903 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290564902 2023-02-13 12:15:34,904 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 12:15:34,908 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:34,909 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 12:15:34,910 DEBUG [RS:1;jenkins-hbase16:33961] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3f921883, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:15:34,910 DEBUG [RS:2;jenkins-hbase16:45345] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@322e6e63, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:15:34,910 DEBUG [RS:0;jenkins-hbase16:33055] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@444a1eaf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:15:34,911 DEBUG [RS:1;jenkins-hbase16:33961] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3bad9ba9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-02-13 12:15:34,911 DEBUG [RS:2;jenkins-hbase16:45345] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3611ecfe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-02-13 12:15:34,911 DEBUG [RS:0;jenkins-hbase16:33055] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@407582fe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-02-13 12:15:34,914 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 12:15:34,915 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 12:15:34,922 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 12:15:34,922 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 12:15:34,923 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 12:15:34,923 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 12:15:34,923 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:34,925 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 12:15:34,926 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 12:15:34,927 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 12:15:34,930 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 12:15:34,930 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 12:15:34,932 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1676290534931,5,FailOnTimeoutGroup] 2023-02-13 12:15:34,932 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1676290534932,5,FailOnTimeoutGroup] 2023-02-13 12:15:34,932 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:34,932 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 12:15:34,933 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:34,934 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:34,936 DEBUG [RS:1;jenkins-hbase16:33961] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase16:33961 2023-02-13 12:15:34,936 DEBUG [RS:2;jenkins-hbase16:45345] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase16:45345 2023-02-13 12:15:34,939 DEBUG [RS:0;jenkins-hbase16:33055] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase16:33055 2023-02-13 12:15:34,942 INFO [RS:1;jenkins-hbase16:33961] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 12:15:34,942 INFO [RS:1;jenkins-hbase16:33961] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 12:15:34,942 INFO [RS:0;jenkins-hbase16:33055] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 12:15:34,942 INFO [RS:0;jenkins-hbase16:33055] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 12:15:34,942 INFO [RS:2;jenkins-hbase16:45345] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 12:15:34,942 INFO [RS:2;jenkins-hbase16:45345] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 12:15:34,942 DEBUG [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 12:15:34,942 DEBUG [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 12:15:34,942 DEBUG [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 12:15:34,948 INFO [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase16.apache.org,39699,1676290532646 with isa=jenkins-hbase16.apache.org/188.40.62.62:33055, startcode=1676290533708 2023-02-13 12:15:34,948 INFO [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase16.apache.org,39699,1676290532646 with isa=jenkins-hbase16.apache.org/188.40.62.62:33961, startcode=1676290533759 2023-02-13 12:15:34,948 INFO [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase16.apache.org,39699,1676290532646 with isa=jenkins-hbase16.apache.org/188.40.62.62:45345, startcode=1676290533799 2023-02-13 12:15:34,959 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 12:15:34,960 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 12:15:34,961 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8 2023-02-13 12:15:34,981 DEBUG [RS:2;jenkins-hbase16:45345] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 12:15:34,981 DEBUG [RS:1;jenkins-hbase16:33961] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 12:15:34,984 DEBUG [RS:0;jenkins-hbase16:33055] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 12:15:34,984 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:34,988 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 12:15:34,990 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/info 2023-02-13 12:15:34,991 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 12:15:34,994 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:34,994 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 12:15:34,997 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/rep_barrier 2023-02-13 12:15:34,997 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 12:15:34,998 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:34,998 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 12:15:35,001 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/table 2023-02-13 12:15:35,001 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 12:15:35,002 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:35,004 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740 2023-02-13 12:15:35,005 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740 2023-02-13 12:15:35,009 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 12:15:35,011 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 12:15:35,015 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:15:35,016 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59467659, jitterRate=-0.11386282742023468}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 12:15:35,016 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676290534985Initializing all the Stores at 1676290534987 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676290534987Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290534987Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676290534987Cleaning up temporary data from old regions at 1676290535007 (+20 ms)Cleaning up detritus from prior splits at 1676290535007Region opened successfully at 1676290535016 (+9 ms) 2023-02-13 12:15:35,017 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 12:15:35,017 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 12:15:35,017 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 12:15:35,017 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 12:15:35,017 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 12:15:35,018 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 12:15:35,018 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676290535017Disabling compacts and flushes for region at 1676290535017Disabling writes for close at 1676290535017Writing region close event to WAL at 1676290535018 (+1 ms)Closed at 1676290535018 2023-02-13 12:15:35,029 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:35,029 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 12:15:35,039 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 12:15:35,042 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:41175, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 12:15:35,042 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:55577, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 12:15:35,042 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45643, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 12:15:35,054 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39699] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:35,054 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:35,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39699] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:35,055 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 12:15:35,058 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 12:15:35,069 DEBUG [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8 2023-02-13 12:15:35,069 DEBUG [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8 2023-02-13 12:15:35,070 DEBUG [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:44273 2023-02-13 12:15:35,070 DEBUG [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 12:15:35,069 DEBUG [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8 2023-02-13 12:15:35,069 DEBUG [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:44273 2023-02-13 12:15:35,070 DEBUG [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:44273 2023-02-13 12:15:35,070 DEBUG [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 12:15:35,070 DEBUG [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 12:15:35,081 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 12:15:35,114 DEBUG [RS:2;jenkins-hbase16:45345] zookeeper.ZKUtil(162): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:35,114 WARN [RS:2;jenkins-hbase16:45345] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 12:15:35,114 INFO [RS:2;jenkins-hbase16:45345] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 12:15:35,115 DEBUG [RS:1;jenkins-hbase16:33961] zookeeper.ZKUtil(162): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:35,115 DEBUG [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:35,114 DEBUG [RS:0;jenkins-hbase16:33055] zookeeper.ZKUtil(162): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:35,115 WARN [RS:1;jenkins-hbase16:33961] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 12:15:35,115 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,33961,1676290533759] 2023-02-13 12:15:35,116 INFO [RS:1;jenkins-hbase16:33961] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 12:15:35,116 WARN [RS:0;jenkins-hbase16:33055] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 12:15:35,117 DEBUG [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:35,117 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,33055,1676290533708] 2023-02-13 12:15:35,118 INFO [RS:0;jenkins-hbase16:33055] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 12:15:35,118 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,45345,1676290533799] 2023-02-13 12:15:35,118 DEBUG [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:35,130 DEBUG [RS:0;jenkins-hbase16:33055] zookeeper.ZKUtil(162): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:35,130 DEBUG [RS:1;jenkins-hbase16:33961] zookeeper.ZKUtil(162): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:35,130 DEBUG [RS:2;jenkins-hbase16:45345] zookeeper.ZKUtil(162): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:35,131 DEBUG [RS:0;jenkins-hbase16:33055] zookeeper.ZKUtil(162): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:35,131 DEBUG [RS:1;jenkins-hbase16:33961] zookeeper.ZKUtil(162): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:35,131 DEBUG [RS:2;jenkins-hbase16:45345] zookeeper.ZKUtil(162): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:35,131 DEBUG [RS:0;jenkins-hbase16:33055] zookeeper.ZKUtil(162): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:35,131 DEBUG [RS:2;jenkins-hbase16:45345] zookeeper.ZKUtil(162): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:35,131 DEBUG [RS:1;jenkins-hbase16:33961] zookeeper.ZKUtil(162): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:35,141 DEBUG [RS:1;jenkins-hbase16:33961] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 12:15:35,141 DEBUG [RS:0;jenkins-hbase16:33055] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 12:15:35,141 DEBUG [RS:2;jenkins-hbase16:45345] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 12:15:35,149 INFO [RS:2;jenkins-hbase16:45345] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 12:15:35,149 INFO [RS:0;jenkins-hbase16:33055] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 12:15:35,150 INFO [RS:1;jenkins-hbase16:33961] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 12:15:35,169 INFO [RS:1;jenkins-hbase16:33961] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 12:15:35,169 INFO [RS:0;jenkins-hbase16:33055] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 12:15:35,169 INFO [RS:2;jenkins-hbase16:45345] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 12:15:35,173 INFO [RS:1;jenkins-hbase16:33961] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:15:35,173 INFO [RS:0;jenkins-hbase16:33055] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:15:35,174 INFO [RS:1;jenkins-hbase16:33961] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,174 INFO [RS:0;jenkins-hbase16:33055] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,173 INFO [RS:2;jenkins-hbase16:45345] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:15:35,174 INFO [RS:2;jenkins-hbase16:45345] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,175 INFO [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 12:15:35,175 INFO [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 12:15:35,175 INFO [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 12:15:35,182 INFO [RS:1;jenkins-hbase16:33961] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,182 INFO [RS:2;jenkins-hbase16:45345] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,182 INFO [RS:0;jenkins-hbase16:33055] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,182 DEBUG [RS:1;jenkins-hbase16:33961] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,182 DEBUG [RS:2;jenkins-hbase16:45345] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:1;jenkins-hbase16:33961] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:2;jenkins-hbase16:45345] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:0;jenkins-hbase16:33055] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:2;jenkins-hbase16:45345] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:0;jenkins-hbase16:33055] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:2;jenkins-hbase16:45345] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:0;jenkins-hbase16:33055] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:2;jenkins-hbase16:45345] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:0;jenkins-hbase16:33055] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:2;jenkins-hbase16:45345] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-02-13 12:15:35,183 DEBUG [RS:0;jenkins-hbase16:33055] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:2;jenkins-hbase16:45345] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:0;jenkins-hbase16:33055] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-02-13 12:15:35,183 DEBUG [RS:1;jenkins-hbase16:33961] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:2;jenkins-hbase16:45345] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,183 DEBUG [RS:0;jenkins-hbase16:33055] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,184 DEBUG [RS:2;jenkins-hbase16:45345] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,184 DEBUG [RS:0;jenkins-hbase16:33055] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,184 DEBUG [RS:2;jenkins-hbase16:45345] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,184 DEBUG [RS:1;jenkins-hbase16:33961] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,184 DEBUG [RS:0;jenkins-hbase16:33055] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,185 DEBUG [RS:1;jenkins-hbase16:33961] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,185 DEBUG [RS:0;jenkins-hbase16:33055] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,185 DEBUG [RS:1;jenkins-hbase16:33961] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-02-13 12:15:35,185 DEBUG [RS:1;jenkins-hbase16:33961] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,185 DEBUG [RS:1;jenkins-hbase16:33961] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,185 DEBUG [RS:1;jenkins-hbase16:33961] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,185 DEBUG [RS:1;jenkins-hbase16:33961] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:15:35,186 INFO [RS:0;jenkins-hbase16:33055] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,186 INFO [RS:0;jenkins-hbase16:33055] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,186 INFO [RS:0;jenkins-hbase16:33055] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,186 INFO [RS:2;jenkins-hbase16:45345] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,186 INFO [RS:2;jenkins-hbase16:45345] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,187 INFO [RS:2;jenkins-hbase16:45345] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,187 INFO [RS:1;jenkins-hbase16:33961] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,187 INFO [RS:1;jenkins-hbase16:33961] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,187 INFO [RS:1;jenkins-hbase16:33961] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,199 INFO [RS:2;jenkins-hbase16:45345] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 12:15:35,199 INFO [RS:0;jenkins-hbase16:33055] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 12:15:35,199 INFO [RS:1;jenkins-hbase16:33961] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 12:15:35,202 INFO [RS:2;jenkins-hbase16:45345] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,45345,1676290533799-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,202 INFO [RS:0;jenkins-hbase16:33055] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,33055,1676290533708-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,202 INFO [RS:1;jenkins-hbase16:33961] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,33961,1676290533759-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,210 DEBUG [jenkins-hbase16:39699] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 12:15:35,213 DEBUG [jenkins-hbase16:39699] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:15:35,218 DEBUG [jenkins-hbase16:39699] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:15:35,218 DEBUG [jenkins-hbase16:39699] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:15:35,218 DEBUG [jenkins-hbase16:39699] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:15:35,218 DEBUG [jenkins-hbase16:39699] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:15:35,221 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,33961,1676290533759, state=OPENING 2023-02-13 12:15:35,223 INFO [RS:2;jenkins-hbase16:45345] regionserver.Replication(203): jenkins-hbase16.apache.org,45345,1676290533799 started 2023-02-13 12:15:35,223 INFO [RS:1;jenkins-hbase16:33961] regionserver.Replication(203): jenkins-hbase16.apache.org,33961,1676290533759 started 2023-02-13 12:15:35,223 INFO [RS:0;jenkins-hbase16:33055] regionserver.Replication(203): jenkins-hbase16.apache.org,33055,1676290533708 started 2023-02-13 12:15:35,223 INFO [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1638): Serving as jenkins-hbase16.apache.org,33961,1676290533759, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:33961, sessionid=0x10052d110940002 2023-02-13 12:15:35,223 INFO [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(1638): Serving as jenkins-hbase16.apache.org,45345,1676290533799, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:45345, sessionid=0x10052d110940003 2023-02-13 12:15:35,223 INFO [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1638): Serving as jenkins-hbase16.apache.org,33055,1676290533708, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:33055, sessionid=0x10052d110940001 2023-02-13 12:15:35,223 DEBUG [RS:2;jenkins-hbase16:45345] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 12:15:35,223 DEBUG [RS:0;jenkins-hbase16:33055] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 12:15:35,223 DEBUG [RS:2;jenkins-hbase16:45345] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:35,223 DEBUG [RS:1;jenkins-hbase16:33961] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 12:15:35,224 DEBUG [RS:2;jenkins-hbase16:45345] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,45345,1676290533799' 2023-02-13 12:15:35,223 DEBUG [RS:0;jenkins-hbase16:33055] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:35,224 DEBUG [RS:2;jenkins-hbase16:45345] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:35,224 DEBUG [RS:1;jenkins-hbase16:33961] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:35,224 DEBUG [RS:0;jenkins-hbase16:33055] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,33055,1676290533708' 2023-02-13 12:15:35,225 DEBUG [RS:0;jenkins-hbase16:33055] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:35,225 DEBUG [RS:1;jenkins-hbase16:33961] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,33961,1676290533759' 2023-02-13 12:15:35,225 DEBUG [RS:1;jenkins-hbase16:33961] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:35,225 DEBUG [RS:0;jenkins-hbase16:33055] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:35,225 DEBUG [RS:2;jenkins-hbase16:45345] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:35,225 DEBUG [RS:1;jenkins-hbase16:33961] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:35,226 DEBUG [RS:0;jenkins-hbase16:33055] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 12:15:35,226 DEBUG [RS:1;jenkins-hbase16:33961] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 12:15:35,226 DEBUG [RS:0;jenkins-hbase16:33055] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 12:15:35,226 DEBUG [RS:2;jenkins-hbase16:45345] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 12:15:35,226 DEBUG [RS:0;jenkins-hbase16:33055] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:35,226 DEBUG [RS:1;jenkins-hbase16:33961] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 12:15:35,226 DEBUG [RS:0;jenkins-hbase16:33055] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,33055,1676290533708' 2023-02-13 12:15:35,226 DEBUG [RS:0;jenkins-hbase16:33055] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 12:15:35,226 DEBUG [RS:2;jenkins-hbase16:45345] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 12:15:35,226 DEBUG [RS:1;jenkins-hbase16:33961] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:35,227 DEBUG [RS:2;jenkins-hbase16:45345] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:35,227 DEBUG [RS:0;jenkins-hbase16:33055] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 12:15:35,227 DEBUG [RS:2;jenkins-hbase16:45345] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,45345,1676290533799' 2023-02-13 12:15:35,227 DEBUG [RS:2;jenkins-hbase16:45345] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 12:15:35,227 DEBUG [RS:1;jenkins-hbase16:33961] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,33961,1676290533759' 2023-02-13 12:15:35,227 DEBUG [RS:1;jenkins-hbase16:33961] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 12:15:35,227 DEBUG [RS:0;jenkins-hbase16:33055] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 12:15:35,227 INFO [RS:0;jenkins-hbase16:33055] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 12:15:35,227 INFO [RS:0;jenkins-hbase16:33055] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 12:15:35,227 DEBUG [RS:2;jenkins-hbase16:45345] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 12:15:35,228 DEBUG [RS:1;jenkins-hbase16:33961] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 12:15:35,228 DEBUG [RS:2;jenkins-hbase16:45345] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 12:15:35,228 DEBUG [RS:1;jenkins-hbase16:33961] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 12:15:35,228 INFO [RS:2;jenkins-hbase16:45345] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 12:15:35,229 INFO [RS:2;jenkins-hbase16:45345] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 12:15:35,229 INFO [RS:1;jenkins-hbase16:33961] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 12:15:35,229 INFO [RS:1;jenkins-hbase16:33961] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 12:15:35,242 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 12:15:35,251 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:15:35,251 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 12:15:35,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:15:35,340 INFO [RS:2;jenkins-hbase16:45345] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C45345%2C1676290533799, suffix=, logDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,45345,1676290533799, archiveDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/oldWALs, maxLogs=32 2023-02-13 12:15:35,340 INFO [RS:0;jenkins-hbase16:33055] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C33055%2C1676290533708, suffix=, logDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,33055,1676290533708, archiveDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/oldWALs, maxLogs=32 2023-02-13 12:15:35,340 INFO [RS:1;jenkins-hbase16:33961] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C33961%2C1676290533759, suffix=, logDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,33961,1676290533759, archiveDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/oldWALs, maxLogs=32 2023-02-13 12:15:35,361 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33121,DS-76042f15-cd3b-4856-a50f-9b89070b594e,DISK] 2023-02-13 12:15:35,361 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38573,DS-16a8c210-c13d-4264-abac-6466dde30377,DISK] 2023-02-13 12:15:35,363 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34415,DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f,DISK] 2023-02-13 12:15:35,370 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33121,DS-76042f15-cd3b-4856-a50f-9b89070b594e,DISK] 2023-02-13 12:15:35,370 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34415,DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f,DISK] 2023-02-13 12:15:35,370 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38573,DS-16a8c210-c13d-4264-abac-6466dde30377,DISK] 2023-02-13 12:15:35,383 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33121,DS-76042f15-cd3b-4856-a50f-9b89070b594e,DISK] 2023-02-13 12:15:35,383 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38573,DS-16a8c210-c13d-4264-abac-6466dde30377,DISK] 2023-02-13 12:15:35,383 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34415,DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f,DISK] 2023-02-13 12:15:35,385 INFO [RS:0;jenkins-hbase16:33055] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,33055,1676290533708/jenkins-hbase16.apache.org%2C33055%2C1676290533708.1676290535344 2023-02-13 12:15:35,387 DEBUG [RS:0;jenkins-hbase16:33055] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38573,DS-16a8c210-c13d-4264-abac-6466dde30377,DISK], DatanodeInfoWithStorage[127.0.0.1:33121,DS-76042f15-cd3b-4856-a50f-9b89070b594e,DISK], DatanodeInfoWithStorage[127.0.0.1:34415,DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f,DISK]] 2023-02-13 12:15:35,394 INFO [RS:2;jenkins-hbase16:45345] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,45345,1676290533799/jenkins-hbase16.apache.org%2C45345%2C1676290533799.1676290535345 2023-02-13 12:15:35,394 INFO [RS:1;jenkins-hbase16:33961] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,33961,1676290533759/jenkins-hbase16.apache.org%2C33961%2C1676290533759.1676290535344 2023-02-13 12:15:35,394 DEBUG [RS:2;jenkins-hbase16:45345] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34415,DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f,DISK], DatanodeInfoWithStorage[127.0.0.1:33121,DS-76042f15-cd3b-4856-a50f-9b89070b594e,DISK], DatanodeInfoWithStorage[127.0.0.1:38573,DS-16a8c210-c13d-4264-abac-6466dde30377,DISK]] 2023-02-13 12:15:35,394 DEBUG [RS:1;jenkins-hbase16:33961] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34415,DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f,DISK], DatanodeInfoWithStorage[127.0.0.1:33121,DS-76042f15-cd3b-4856-a50f-9b89070b594e,DISK], DatanodeInfoWithStorage[127.0.0.1:38573,DS-16a8c210-c13d-4264-abac-6466dde30377,DISK]] 2023-02-13 12:15:35,436 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:35,438 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 12:15:35,441 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45972, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 12:15:35,455 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 12:15:35,456 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 12:15:35,460 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C33961%2C1676290533759.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,33961,1676290533759, archiveDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/oldWALs, maxLogs=32 2023-02-13 12:15:35,481 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33121,DS-76042f15-cd3b-4856-a50f-9b89070b594e,DISK] 2023-02-13 12:15:35,481 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34415,DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f,DISK] 2023-02-13 12:15:35,483 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38573,DS-16a8c210-c13d-4264-abac-6466dde30377,DISK] 2023-02-13 12:15:35,492 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/WALs/jenkins-hbase16.apache.org,33961,1676290533759/jenkins-hbase16.apache.org%2C33961%2C1676290533759.meta.1676290535462.meta 2023-02-13 12:15:35,496 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34415,DS-6f925c51-f2cd-4564-a69e-14eaed0fca8f,DISK], DatanodeInfoWithStorage[127.0.0.1:33121,DS-76042f15-cd3b-4856-a50f-9b89070b594e,DISK], DatanodeInfoWithStorage[127.0.0.1:38573,DS-16a8c210-c13d-4264-abac-6466dde30377,DISK]] 2023-02-13 12:15:35,496 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:15:35,500 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:35,501 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 12:15:35,519 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 12:15:35,521 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 12:15:35,524 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 12:15:35,525 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:35,525 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 12:15:35,525 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 12:15:35,528 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 12:15:35,529 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/info 2023-02-13 12:15:35,529 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/info 2023-02-13 12:15:35,530 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 12:15:35,531 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:35,531 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 12:15:35,532 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/rep_barrier 2023-02-13 12:15:35,533 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/rep_barrier 2023-02-13 12:15:35,533 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 12:15:35,534 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:35,534 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 12:15:35,536 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/table 2023-02-13 12:15:35,536 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/table 2023-02-13 12:15:35,536 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 12:15:35,537 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:35,539 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740 2023-02-13 12:15:35,542 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740 2023-02-13 12:15:35,546 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 12:15:35,548 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 12:15:35,549 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62872775, jitterRate=-0.06312264502048492}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 12:15:35,550 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676290535525Writing region info on filesystem at 1676290535525Initializing all the Stores at 1676290535527 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676290535527Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290535527Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676290535527Cleaning up temporary data from old regions at 1676290535544 (+17 ms)Cleaning up detritus from prior splits at 1676290535545 (+1 ms)Running coprocessor post-open hooks at 1676290535549 (+4 ms)Region opened successfully at 1676290535550 (+1 ms) 2023-02-13 12:15:35,559 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676290535429 2023-02-13 12:15:35,577 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 12:15:35,578 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 12:15:35,579 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,33961,1676290533759, state=OPEN 2023-02-13 12:15:35,590 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 12:15:35,590 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 12:15:35,594 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 12:15:35,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,33961,1676290533759 in 335 msec 2023-02-13 12:15:35,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 12:15:35,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 558 msec 2023-02-13 12:15:35,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 795 msec 2023-02-13 12:15:35,607 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676290533856, completionTime=-1 2023-02-13 12:15:35,607 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 12:15:35,607 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:35,666 DEBUG [hconnection-0x3fbde3a3-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:15:35,669 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45988, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:15:35,685 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:35,685 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290595685 2023-02-13 12:15:35,685 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290655685 2023-02-13 12:15:35,685 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 78 msec 2023-02-13 12:15:35,722 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,39699,1676290532646-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,722 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,39699,1676290532646-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,722 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,39699,1676290532646-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,724 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase16:39699, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,724 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 12:15:35,731 DEBUG [master/jenkins-hbase16:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 12:15:35,744 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 12:15:35,745 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 12:15:35,754 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:35,757 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:15:35,759 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:15:35,782 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/hbase/namespace/97396b7f803d52a54325415167dadd83 2023-02-13 12:15:35,784 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/hbase/namespace/97396b7f803d52a54325415167dadd83 empty. 2023-02-13 12:15:35,785 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/hbase/namespace/97396b7f803d52a54325415167dadd83 2023-02-13 12:15:35,785 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 12:15:35,849 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 12:15:35,851 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 97396b7f803d52a54325415167dadd83, NAME => 'hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:15:35,870 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:35,870 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 97396b7f803d52a54325415167dadd83, disabling compactions & flushes 2023-02-13 12:15:35,870 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:15:35,870 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:15:35,870 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. after waiting 0 ms 2023-02-13 12:15:35,871 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:15:35,871 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:15:35,871 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 97396b7f803d52a54325415167dadd83: Waiting for close lock at 1676290535870Disabling compacts and flushes for region at 1676290535870Disabling writes for close at 1676290535871 (+1 ms)Writing region close event to WAL at 1676290535871Closed at 1676290535871 2023-02-13 12:15:35,876 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:15:35,890 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676290535879"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290535879"}]},"ts":"1676290535879"} 2023-02-13 12:15:35,911 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:15:35,913 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:15:35,917 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290535913"}]},"ts":"1676290535913"} 2023-02-13 12:15:35,921 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 12:15:35,943 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:15:35,944 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:15:35,944 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:15:35,944 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:15:35,944 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:15:35,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=97396b7f803d52a54325415167dadd83, ASSIGN}] 2023-02-13 12:15:35,954 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=97396b7f803d52a54325415167dadd83, ASSIGN 2023-02-13 12:15:35,955 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=97396b7f803d52a54325415167dadd83, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:15:36,109 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:15:36,110 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=97396b7f803d52a54325415167dadd83, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:36,111 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676290536110"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290536110"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290536110"}]},"ts":"1676290536110"} 2023-02-13 12:15:36,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 97396b7f803d52a54325415167dadd83, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:15:36,274 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:36,274 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 12:15:36,282 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45954, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 12:15:36,292 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:15:36,294 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 97396b7f803d52a54325415167dadd83, NAME => 'hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:15:36,296 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:36,296 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 97396b7f803d52a54325415167dadd83 2023-02-13 12:15:36,296 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:36,297 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 97396b7f803d52a54325415167dadd83 2023-02-13 12:15:36,297 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 97396b7f803d52a54325415167dadd83 2023-02-13 12:15:36,299 INFO [StoreOpener-97396b7f803d52a54325415167dadd83-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 97396b7f803d52a54325415167dadd83 2023-02-13 12:15:36,301 DEBUG [StoreOpener-97396b7f803d52a54325415167dadd83-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/namespace/97396b7f803d52a54325415167dadd83/info 2023-02-13 12:15:36,301 DEBUG [StoreOpener-97396b7f803d52a54325415167dadd83-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/namespace/97396b7f803d52a54325415167dadd83/info 2023-02-13 12:15:36,301 INFO [StoreOpener-97396b7f803d52a54325415167dadd83-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 97396b7f803d52a54325415167dadd83 columnFamilyName info 2023-02-13 12:15:36,302 INFO [StoreOpener-97396b7f803d52a54325415167dadd83-1] regionserver.HStore(310): Store=97396b7f803d52a54325415167dadd83/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:36,304 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/namespace/97396b7f803d52a54325415167dadd83 2023-02-13 12:15:36,305 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/namespace/97396b7f803d52a54325415167dadd83 2023-02-13 12:15:36,310 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 97396b7f803d52a54325415167dadd83 2023-02-13 12:15:36,314 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/namespace/97396b7f803d52a54325415167dadd83/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:15:36,315 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 97396b7f803d52a54325415167dadd83; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73493659, jitterRate=0.09514085948467255}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:15:36,315 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 97396b7f803d52a54325415167dadd83: Running coprocessor pre-open hook at 1676290536297Writing region info on filesystem at 1676290536297Initializing all the Stores at 1676290536298 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676290536298Cleaning up temporary data from old regions at 1676290536306 (+8 ms)Cleaning up detritus from prior splits at 1676290536306Running coprocessor post-open hooks at 1676290536315 (+9 ms)Region opened successfully at 1676290536315 2023-02-13 12:15:36,317 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83., pid=6, masterSystemTime=1676290536274 2023-02-13 12:15:36,322 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:15:36,322 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:15:36,324 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=97396b7f803d52a54325415167dadd83, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:36,324 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676290536323"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290536323"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290536323"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290536323"}]},"ts":"1676290536323"} 2023-02-13 12:15:36,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 12:15:36,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 97396b7f803d52a54325415167dadd83, server=jenkins-hbase16.apache.org,33055,1676290533708 in 209 msec 2023-02-13 12:15:36,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 12:15:36,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=97396b7f803d52a54325415167dadd83, ASSIGN in 383 msec 2023-02-13 12:15:36,335 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:15:36,335 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290536335"}]},"ts":"1676290536335"} 2023-02-13 12:15:36,337 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 12:15:36,375 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 12:15:36,376 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:15:36,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 630 msec 2023-02-13 12:15:36,384 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 12:15:36,384 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:15:36,406 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:15:36,409 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45958, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:15:36,421 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:36,442 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 12:15:36,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 41 msec 2023-02-13 12:15:36,464 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:36,484 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 12:15:36,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 32 msec 2023-02-13 12:15:36,526 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 12:15:36,543 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 12:15:36,546 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.660sec 2023-02-13 12:15:36,550 INFO [master/jenkins-hbase16:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 12:15:36,553 INFO [master/jenkins-hbase16:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 12:15:36,553 INFO [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 12:15:36,555 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,39699,1676290532646-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 12:15:36,557 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,39699,1676290532646-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 12:15:36,566 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 12:15:36,635 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ReadOnlyZKClient(139): Connect 0x42379693 to 127.0.0.1:54988 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:15:36,649 DEBUG [Listener at localhost.localdomain/46087] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c9c45f5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:15:36,666 DEBUG [hconnection-0x3e7d9739-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:15:36,680 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45990, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:15:36,689 INFO [Listener at localhost.localdomain/46087] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase16.apache.org,39699,1676290532646 2023-02-13 12:15:36,716 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=361, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=269, ProcessCount=167, AvailableMemoryMB=6082 2023-02-13 12:15:36,721 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:15:36,731 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:15:36,734 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:40934, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:15:36,759 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-13 12:15:36,759 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:15:36,761 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 12:15:36,768 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:36,770 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:15:36,773 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:15:36,776 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:36,777 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d empty. 2023-02-13 12:15:36,779 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:36,779 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 12:15:36,802 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 12:15:36,803 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => b5548b9ced331770542633d75bc36e0d, NAME => 'testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:15:36,824 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:36,824 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing b5548b9ced331770542633d75bc36e0d, disabling compactions & flushes 2023-02-13 12:15:36,825 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:36,825 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:36,825 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. after waiting 0 ms 2023-02-13 12:15:36,825 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:36,825 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:36,825 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for b5548b9ced331770542633d75bc36e0d: Waiting for close lock at 1676290536824Disabling compacts and flushes for region at 1676290536824Disabling writes for close at 1676290536825 (+1 ms)Writing region close event to WAL at 1676290536825Closed at 1676290536825 2023-02-13 12:15:36,829 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:15:36,831 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676290536831"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290536831"}]},"ts":"1676290536831"} 2023-02-13 12:15:36,834 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:15:36,835 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:15:36,836 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290536835"}]},"ts":"1676290536835"} 2023-02-13 12:15:36,838 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-13 12:15:36,857 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:15:36,859 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:15:36,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:15:36,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:15:36,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:15:36,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b5548b9ced331770542633d75bc36e0d, ASSIGN}] 2023-02-13 12:15:36,865 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b5548b9ced331770542633d75bc36e0d, ASSIGN 2023-02-13 12:15:36,867 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b5548b9ced331770542633d75bc36e0d, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:15:37,018 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:15:37,020 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=b5548b9ced331770542633d75bc36e0d, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,020 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676290537019"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290537019"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290537019"}]},"ts":"1676290537019"} 2023-02-13 12:15:37,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure b5548b9ced331770542633d75bc36e0d, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:15:37,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:37,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b5548b9ced331770542633d75bc36e0d, NAME => 'testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:15:37,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:37,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:37,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:37,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:37,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:37,197 INFO [StoreOpener-b5548b9ced331770542633d75bc36e0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:37,199 DEBUG [StoreOpener-b5548b9ced331770542633d75bc36e0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1 2023-02-13 12:15:37,199 DEBUG [StoreOpener-b5548b9ced331770542633d75bc36e0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1 2023-02-13 12:15:37,200 INFO [StoreOpener-b5548b9ced331770542633d75bc36e0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5548b9ced331770542633d75bc36e0d columnFamilyName cf1 2023-02-13 12:15:37,200 INFO [StoreOpener-b5548b9ced331770542633d75bc36e0d-1] regionserver.HStore(310): Store=b5548b9ced331770542633d75bc36e0d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:37,201 INFO [StoreOpener-b5548b9ced331770542633d75bc36e0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:37,203 DEBUG [StoreOpener-b5548b9ced331770542633d75bc36e0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2 2023-02-13 12:15:37,203 DEBUG [StoreOpener-b5548b9ced331770542633d75bc36e0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2 2023-02-13 12:15:37,203 INFO [StoreOpener-b5548b9ced331770542633d75bc36e0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5548b9ced331770542633d75bc36e0d columnFamilyName cf2 2023-02-13 12:15:37,204 INFO [StoreOpener-b5548b9ced331770542633d75bc36e0d-1] regionserver.HStore(310): Store=b5548b9ced331770542633d75bc36e0d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:37,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:37,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:37,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:15:37,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:37,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:15:37,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened b5548b9ced331770542633d75bc36e0d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63495491, jitterRate=-0.05384345352649689}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:15:37,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for b5548b9ced331770542633d75bc36e0d: Running coprocessor pre-open hook at 1676290537195Writing region info on filesystem at 1676290537195Initializing all the Stores at 1676290537197 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290537197Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290537197Cleaning up temporary data from old regions at 1676290537208 (+11 ms)Cleaning up detritus from prior splits at 1676290537208Running coprocessor post-open hooks at 1676290537215 (+7 ms)Region opened successfully at 1676290537215 2023-02-13 12:15:37,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d., pid=11, masterSystemTime=1676290537181 2023-02-13 12:15:37,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:37,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:37,220 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=b5548b9ced331770542633d75bc36e0d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,221 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676290537220"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290537220"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290537220"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290537220"}]},"ts":"1676290537220"} 2023-02-13 12:15:37,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 12:15:37,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure b5548b9ced331770542633d75bc36e0d, server=jenkins-hbase16.apache.org,33055,1676290533708 in 196 msec 2023-02-13 12:15:37,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 12:15:37,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b5548b9ced331770542633d75bc36e0d, ASSIGN in 368 msec 2023-02-13 12:15:37,233 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:15:37,233 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290537233"}]},"ts":"1676290537233"} 2023-02-13 12:15:37,235 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-13 12:15:37,276 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:15:37,281 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 510 msec 2023-02-13 12:15:37,524 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:15:37,529 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45960, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:15:37,550 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:37,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,557 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:37,557 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:37,557 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 12:15:37,557 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 12:15:37,558 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,558 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,592 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:37,593 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:37,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:37,592 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:37,592 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:37,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:37,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:37,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:37,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:37,593 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,593 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,596 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,598 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:37,599 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:37,599 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:37,599 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:37,599 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:37,600 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:37,600 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:37,600 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:37,601 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:37,601 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:37,601 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:37,602 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:37,602 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:37,602 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:37,602 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:37,602 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:37,603 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. started... 2023-02-13 12:15:37,605 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b5548b9ced331770542633d75bc36e0d 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 12:15:37,609 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,609 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,609 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,609 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:37,609 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,609 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:37,670 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/dd75ef112ba9482782912780eb772561 2023-02-13 12:15:37,730 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/08b7e875196a481b8275cfed87a2fb22 2023-02-13 12:15:37,743 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/dd75ef112ba9482782912780eb772561 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561 2023-02-13 12:15:37,754 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:15:37,758 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/08b7e875196a481b8275cfed87a2fb22 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22 2023-02-13 12:15:37,768 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:15:37,770 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b5548b9ced331770542633d75bc36e0d in 165ms, sequenceid=6, compaction requested=false 2023-02-13 12:15:37,772 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:37,775 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b5548b9ced331770542633d75bc36e0d: 2023-02-13 12:15:37,775 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:37,775 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:37,776 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:37,776 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,776 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:37,776 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:37,784 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,784 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:37,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:37,785 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,785 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:37,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:37,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:37,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:37,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:37,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 12:15:37,788 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 12:15:37,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a1e6f5b[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:37,788 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,817 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,817 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,817 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,818 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,818 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:37,818 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:37,818 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:37,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,818 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:37,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,818 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:37,819 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:37,819 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:37,819 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:37,819 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:37,819 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:37,833 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:37,833 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:37,833 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:37,842 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:37,842 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:37,842 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:37,842 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:37,851 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:37,851 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:37,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:37,851 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:37,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:37,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:37,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:37,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:37,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:37,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:37,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:15:37,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:15:37,858 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 12:15:37,858 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:37,858 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:37,858 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,858 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:37,867 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,867 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,867 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:37,867 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:37,867 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,867 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:37,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:37,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,868 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,868 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:37,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:37,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:37,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:37,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:37,869 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:37,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:37,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:37,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:37,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,869 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:37,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:37,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:37,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:37,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:37,872 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,872 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:37,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,873 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:37,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:37,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:37,892 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:37,892 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:37,892 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:37,892 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:37,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:37,893 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:37,893 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:37,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:37,893 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:37,893 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:37,893 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:37,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:37,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:37,893 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:37,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:37,893 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:37,893 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:37,894 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,894 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:37,894 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:37,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:37,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:37,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:37,894 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,895 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:37,895 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:37,895 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,895 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:37,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:37,896 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:37,896 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:37,996 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:38,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:38,034 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:38,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:38,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,037 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:38,037 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,038 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 12:15:38,038 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 12:15:38,038 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,038 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,048 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,048 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,048 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,048 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,048 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,049 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:38,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,050 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:38,050 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:38,050 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,050 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,051 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:38,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,051 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,051 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:38,052 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:38,051 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:38,052 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,052 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,052 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:38,053 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:38,052 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:38,053 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:38,053 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:38,053 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:38,053 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:38,053 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:38,054 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:38,053 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. started... 2023-02-13 12:15:38,054 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b5548b9ced331770542633d75bc36e0d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:38,064 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,073 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,073 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,073 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:38,074 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,074 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:38,077 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/5762a6a063604969948f9629eb8ba933 2023-02-13 12:15:38,105 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/dd23553b9a3644249a7673f027fac1d0 2023-02-13 12:15:38,116 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/5762a6a063604969948f9629eb8ba933 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933 2023-02-13 12:15:38,126 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:15:38,127 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/dd23553b9a3644249a7673f027fac1d0 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0 2023-02-13 12:15:38,136 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:15:38,137 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b5548b9ced331770542633d75bc36e0d in 83ms, sequenceid=19, compaction requested=false 2023-02-13 12:15:38,137 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b5548b9ced331770542633d75bc36e0d: 2023-02-13 12:15:38,137 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:38,138 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:38,138 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:38,138 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,138 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:38,138 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:38,148 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,148 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:38,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:38,149 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,149 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:38,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:38,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:38,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:38,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 12:15:38,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26f6cbe7[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:38,152 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 12:15:38,152 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,159 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,159 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,159 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,159 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:38,159 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,159 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:38,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,159 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:38,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,160 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:38,160 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:38,160 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:38,160 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:38,160 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:38,160 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:38,160 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:38,168 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,168 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:38,168 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,168 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:38,175 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:38,175 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:38,176 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:38,176 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:38,176 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,177 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:38,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:38,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:38,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:15:38,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:15:38,182 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 12:15:38,182 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:38,182 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:38,182 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,182 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:38,192 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,192 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:38,192 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,192 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,193 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,193 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:38,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,193 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:38,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,193 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,194 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,195 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:38,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,196 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,197 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,197 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,242 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,242 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,242 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,242 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,242 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:38,242 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:38,242 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,242 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,243 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,242 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,242 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,244 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:38,244 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:38,242 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,243 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,245 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,245 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,245 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,245 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,245 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,245 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,246 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,344 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:38,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:38,367 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:38,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:38,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,370 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:38,370 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,371 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 12:15:38,371 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 12:15:38,371 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,372 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,390 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,391 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,391 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:38,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,391 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:38,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:38,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:38,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,392 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,392 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,393 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,393 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:38,393 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,393 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:38,393 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:38,395 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,395 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,395 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:38,395 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,395 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:38,396 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,395 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:38,397 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,397 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:38,398 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:38,398 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:38,396 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:38,398 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:38,398 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:38,397 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:38,398 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:38,398 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:38,398 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:38,398 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. started... 2023-02-13 12:15:38,399 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b5548b9ced331770542633d75bc36e0d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:38,409 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,417 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,417 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,417 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:38,423 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,423 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:38,427 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/417a65eaac054a87bde5350a6a03da72 2023-02-13 12:15:38,449 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/c068274c957b4a6dafcc96e6992d674b 2023-02-13 12:15:38,459 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/417a65eaac054a87bde5350a6a03da72 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72 2023-02-13 12:15:38,467 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:15:38,469 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/c068274c957b4a6dafcc96e6992d674b as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b 2023-02-13 12:15:38,477 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:15:38,479 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b5548b9ced331770542633d75bc36e0d in 81ms, sequenceid=32, compaction requested=true 2023-02-13 12:15:38,479 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b5548b9ced331770542633d75bc36e0d: 2023-02-13 12:15:38,479 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:38,479 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:38,480 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:38,480 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,480 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:38,480 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:38,489 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,489 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:38,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:38,490 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,490 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:38,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:38,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:38,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:38,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 12:15:38,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9eefc12[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:38,492 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 12:15:38,492 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,501 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,501 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,501 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,501 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:38,501 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:38,501 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:38,501 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,501 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:38,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,502 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:38,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,502 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:38,502 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:38,502 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:38,502 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:38,502 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:38,511 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:38,511 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,511 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:38,523 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:38,523 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,523 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,523 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:38,523 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:38,524 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,524 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:38,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:38,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:38,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:38,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:38,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:38,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:15:38,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:15:38,537 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 12:15:38,537 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:38,537 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:38,537 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,537 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:38,548 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,548 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:38,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:38,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,548 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,548 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,548 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,548 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,548 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,549 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:38,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,550 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,550 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:38,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:38,552 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,553 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,553 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:38,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:38,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,565 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,565 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,566 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:38,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,566 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,566 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,566 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,566 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,566 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,566 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,666 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:38,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:38,697 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:38,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:38,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,699 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:38,699 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,700 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 12:15:38,700 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 12:15:38,701 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,701 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,709 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,710 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,710 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,711 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,711 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,711 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:38,711 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:38,711 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:38,711 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,711 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,711 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,711 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:38,711 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:38,711 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,711 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:38,712 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:38,711 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,712 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:38,712 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:38,713 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:38,713 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:38,712 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:38,713 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:38,713 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:38,713 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:38,714 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:38,714 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:38,714 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. started... 2023-02-13 12:15:38,714 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b5548b9ced331770542633d75bc36e0d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:38,724 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,731 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,731 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,733 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:38,734 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,734 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:38,741 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/b1d1b836d4b046d0a377fc1fbfccea8d 2023-02-13 12:15:38,765 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/8d338657e44f4eec946680459b7a4e02 2023-02-13 12:15:38,773 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/b1d1b836d4b046d0a377fc1fbfccea8d as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d 2023-02-13 12:15:38,779 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:15:38,780 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/8d338657e44f4eec946680459b7a4e02 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02 2023-02-13 12:15:38,788 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:15:38,789 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b5548b9ced331770542633d75bc36e0d in 75ms, sequenceid=45, compaction requested=true 2023-02-13 12:15:38,789 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b5548b9ced331770542633d75bc36e0d: 2023-02-13 12:15:38,789 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:38,789 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:38,790 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:38,790 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,790 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:38,790 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:38,819 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,819 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:38,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:38,819 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,819 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:38,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:38,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:38,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:38,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 12:15:38,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e1703fb[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:38,823 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 12:15:38,823 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,834 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,834 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,834 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,835 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:38,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,835 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:38,835 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:38,835 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:38,835 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,835 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:38,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,835 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:38,836 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:38,835 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:38,836 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:38,836 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:38,842 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:38,842 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,843 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:38,851 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:38,851 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,851 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,851 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:38,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:38,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:38,851 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:38,852 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,852 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:38,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:38,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:15:38,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:15:38,863 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 12:15:38,863 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:38,863 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:38,863 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,863 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:38,873 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:38,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:38,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,873 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:38,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,873 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,874 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,874 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:38,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,876 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,876 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,877 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,884 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,884 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,884 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,884 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,884 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:38,884 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:38,884 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:38,885 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:38,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,884 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,884 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,884 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,885 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:38,886 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:38,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,885 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:38,886 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:38,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:38,885 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,886 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:38,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,886 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:38,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:38,886 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:38,887 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:38,887 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,887 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:38,986 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:38,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:39,018 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:39,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:39,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,021 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:39,021 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:39,021 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 12:15:39,022 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 12:15:39,022 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,022 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,031 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,031 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,031 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,031 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,032 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,032 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:39,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:39,034 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:39,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:39,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,034 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:39,034 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:39,035 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:39,035 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:39,035 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:39,034 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:39,036 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:39,035 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:39,036 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:39,036 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:39,038 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:39,038 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:39,038 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:39,038 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:39,038 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:39,038 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. started... 2023-02-13 12:15:39,039 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b5548b9ced331770542633d75bc36e0d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:39,060 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/98d7b6d98fae444f9feca26096818335 2023-02-13 12:15:39,087 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,088 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/3d27afb320c94d30a5fd4eae210a8b3b 2023-02-13 12:15:39,098 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/98d7b6d98fae444f9feca26096818335 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/98d7b6d98fae444f9feca26096818335 2023-02-13 12:15:39,104 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/98d7b6d98fae444f9feca26096818335, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:15:39,105 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/3d27afb320c94d30a5fd4eae210a8b3b as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/3d27afb320c94d30a5fd4eae210a8b3b 2023-02-13 12:15:39,112 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/3d27afb320c94d30a5fd4eae210a8b3b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:15:39,113 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b5548b9ced331770542633d75bc36e0d in 75ms, sequenceid=58, compaction requested=true 2023-02-13 12:15:39,113 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b5548b9ced331770542633d75bc36e0d: 2023-02-13 12:15:39,113 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:39,113 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:39,114 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:39,114 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,114 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:39,114 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:39,126 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,126 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,126 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:39,134 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,134 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,134 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:39,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:39,134 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:39,135 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,135 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:39,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:39,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:39,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:39,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 12:15:39,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@277ad5d5[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:39,138 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 12:15:39,138 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,148 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,148 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,148 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,149 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,149 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:39,149 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:39,149 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:39,149 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:39,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,149 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:39,150 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:39,150 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:39,150 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:39,150 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:39,150 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:39,159 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:39,159 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:39,159 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:39,167 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:39,168 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:39,167 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:39,168 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:39,167 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,169 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:39,168 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:39,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:39,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:39,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:39,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:39,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:39,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:15:39,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:15:39,178 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 12:15:39,178 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:39,178 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:39,178 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,178 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:39,184 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,184 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:39,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:39,184 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,184 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,184 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,184 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,184 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:39,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,185 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:39,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,185 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,186 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:39,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,188 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,189 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,189 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,220 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,220 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,220 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,220 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,220 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,221 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,221 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:39,221 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:39,221 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,221 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,221 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,221 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:39,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,221 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,222 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,221 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:39,223 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,224 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,223 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:39,224 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,224 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,225 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,225 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,324 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:39,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:39,347 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:39,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:39,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,349 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:39,349 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:39,350 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 12:15:39,350 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 12:15:39,351 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,351 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,365 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,366 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,366 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:39,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:39,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:39,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:39,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,367 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,367 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,367 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,367 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:39,367 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,367 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:39,367 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:39,372 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:39,372 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:39,372 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 12:15:39,372 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,372 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:39,372 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:39,372 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:39,372 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:39,373 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:39,372 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,373 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:39,372 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. started... 2023-02-13 12:15:39,373 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:39,373 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 12:15:39,373 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:39,373 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:39,374 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 12:15:39,374 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:39,374 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:39,374 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b5548b9ced331770542633d75bc36e0d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:39,384 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,384 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,385 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,385 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,385 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:39,385 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:39,398 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/4de66ec55ffd49619300d5454630614b 2023-02-13 12:15:39,424 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/6ce6d57f2d7946c2882e1228e898221c 2023-02-13 12:15:39,432 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/4de66ec55ffd49619300d5454630614b as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/4de66ec55ffd49619300d5454630614b 2023-02-13 12:15:39,439 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/4de66ec55ffd49619300d5454630614b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:15:39,440 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/6ce6d57f2d7946c2882e1228e898221c as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/6ce6d57f2d7946c2882e1228e898221c 2023-02-13 12:15:39,449 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/6ce6d57f2d7946c2882e1228e898221c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:15:39,450 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b5548b9ced331770542633d75bc36e0d in 76ms, sequenceid=71, compaction requested=true 2023-02-13 12:15:39,450 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b5548b9ced331770542633d75bc36e0d: 2023-02-13 12:15:39,451 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:39,451 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:39,451 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:39,451 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,452 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 12:15:39,452 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 12:15:39,459 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,459 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:39,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:39,459 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,460 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:39,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:39,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:39,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:39,462 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 12:15:39,462 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37c38b2e[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:39,462 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 12:15:39,462 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,468 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:39,468 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:39,468 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:39,468 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:39,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,468 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,468 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 12:15:39,468 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:39,468 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 12:15:39,468 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:39,468 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:39,468 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:39,476 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:39,476 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:39,476 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:39,484 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:39,484 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,484 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:39,484 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:39,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:39,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:39,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:39,493 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 12:15:39,493 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:39,493 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 12:15:39,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:39,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:39,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:15:39,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:15:39,504 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 12:15:39,504 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:39,504 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:39,504 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,504 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:39,515 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,515 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:39,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:39,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,515 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,515 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,515 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,515 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:39,515 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,515 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,516 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:39,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,517 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:39,520 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,521 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,521 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:39,534 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:39,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,534 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 12:15:39,535 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:39,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:39,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:39,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:39,535 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,536 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:39,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:39,536 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,536 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:39,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:39,536 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,537 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:39,537 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,537 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:39,537 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,537 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,537 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 12:15:39,637 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:39,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:39,659 DEBUG [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,665 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:39,666 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:39,666 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:39,666 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:39,673 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b5548b9ced331770542633d75bc36e0d, UNASSIGN}] 2023-02-13 12:15:39,675 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b5548b9ced331770542633d75bc36e0d, UNASSIGN 2023-02-13 12:15:39,676 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=b5548b9ced331770542633d75bc36e0d, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:39,676 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676290539676"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290539676"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290539676"}]},"ts":"1676290539676"} 2023-02-13 12:15:39,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure b5548b9ced331770542633d75bc36e0d, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:15:39,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing b5548b9ced331770542633d75bc36e0d, disabling compactions & flushes 2023-02-13 12:15:39,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:39,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:39,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. after waiting 0 ms 2023-02-13 12:15:39,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:39,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing b5548b9ced331770542633d75bc36e0d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 12:15:39,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/d27402b5359b4f6e8f249e7752df33b9 2023-02-13 12:15:39,892 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/41d58eb28fd140e2bbfa14d8ad1a463f 2023-02-13 12:15:39,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf1/d27402b5359b4f6e8f249e7752df33b9 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/d27402b5359b4f6e8f249e7752df33b9 2023-02-13 12:15:39,908 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/d27402b5359b4f6e8f249e7752df33b9, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:15:39,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.tmp/cf2/41d58eb28fd140e2bbfa14d8ad1a463f as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/41d58eb28fd140e2bbfa14d8ad1a463f 2023-02-13 12:15:39,920 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/41d58eb28fd140e2bbfa14d8ad1a463f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:15:39,921 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b5548b9ced331770542633d75bc36e0d in 76ms, sequenceid=82, compaction requested=true 2023-02-13 12:15:39,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 12:15:39,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:15:39,946 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. 2023-02-13 12:15:39,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for b5548b9ced331770542633d75bc36e0d: Waiting for close lock at 1676290539844Running coprocessor pre-close hooks at 1676290539844Disabling compacts and flushes for region at 1676290539844Disabling writes for close at 1676290539845 (+1 ms)Obtaining lock to block concurrent updates at 1676290539845Preparing flush snapshotting stores in b5548b9ced331770542633d75bc36e0d at 1676290539845Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676290539845Flushing stores of testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. at 1676290539846 (+1 ms)Flushing b5548b9ced331770542633d75bc36e0d/cf1: creating writer at 1676290539848 (+2 ms)Flushing b5548b9ced331770542633d75bc36e0d/cf1: appending metadata at 1676290539852 (+4 ms)Flushing b5548b9ced331770542633d75bc36e0d/cf1: closing flushed file at 1676290539852Flushing b5548b9ced331770542633d75bc36e0d/cf2: creating writer at 1676290539875 (+23 ms)Flushing b5548b9ced331770542633d75bc36e0d/cf2: appending metadata at 1676290539878 (+3 ms)Flushing b5548b9ced331770542633d75bc36e0d/cf2: closing flushed file at 1676290539879 (+1 ms)Flushing b5548b9ced331770542633d75bc36e0d/cf1: reopening flushed file at 1676290539901 (+22 ms)Flushing b5548b9ced331770542633d75bc36e0d/cf2: reopening flushed file at 1676290539913 (+12 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b5548b9ced331770542633d75bc36e0d in 76ms, sequenceid=82, compaction requested=true at 1676290539921 (+8 ms)Writing region close event to WAL at 1676290539940 (+19 ms)Running coprocessor post-close hooks at 1676290539945 (+5 ms)Closed at 1676290539946 (+1 ms) 2023-02-13 12:15:39,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,949 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=b5548b9ced331770542633d75bc36e0d, regionState=CLOSED 2023-02-13 12:15:39,950 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676290539949"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290539949"}]},"ts":"1676290539949"} 2023-02-13 12:15:39,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 12:15:39,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure b5548b9ced331770542633d75bc36e0d, server=jenkins-hbase16.apache.org,33055,1676290533708 in 273 msec 2023-02-13 12:15:39,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-13 12:15:39,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b5548b9ced331770542633d75bc36e0d, UNASSIGN in 281 msec 2023-02-13 12:15:39,974 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=b5548b9ced331770542633d75bc36e0d, threads=14 2023-02-13 12:15:39,975 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,975 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/3d27afb320c94d30a5fd4eae210a8b3b for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,976 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/41d58eb28fd140e2bbfa14d8ad1a463f for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,976 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/6ce6d57f2d7946c2882e1228e898221c for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,976 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,976 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,976 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,976 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,976 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/4de66ec55ffd49619300d5454630614b for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,977 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,977 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/98d7b6d98fae444f9feca26096818335 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,977 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,977 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/d27402b5359b4f6e8f249e7752df33b9 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:39,977 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,001 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/3d27afb320c94d30a5fd4eae210a8b3b, top=true 2023-02-13 12:15:40,008 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/6ce6d57f2d7946c2882e1228e898221c, top=true 2023-02-13 12:15:40,017 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/d27402b5359b4f6e8f249e7752df33b9, top=true 2023-02-13 12:15:40,019 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/98d7b6d98fae444f9feca26096818335, top=true 2023-02-13 12:15:40,057 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.splits/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-3d27afb320c94d30a5fd4eae210a8b3b for child: 606b11da264cede32c10b7b752e6175e, parent: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,057 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/3d27afb320c94d30a5fd4eae210a8b3b for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,059 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/4de66ec55ffd49619300d5454630614b, top=true 2023-02-13 12:15:40,065 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/41d58eb28fd140e2bbfa14d8ad1a463f, top=true 2023-02-13 12:15:40,091 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.splits/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-d27402b5359b4f6e8f249e7752df33b9 for child: 606b11da264cede32c10b7b752e6175e, parent: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,091 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/d27402b5359b4f6e8f249e7752df33b9 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,092 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.splits/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-98d7b6d98fae444f9feca26096818335 for child: 606b11da264cede32c10b7b752e6175e, parent: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,093 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/98d7b6d98fae444f9feca26096818335 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,104 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.splits/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-6ce6d57f2d7946c2882e1228e898221c for child: 606b11da264cede32c10b7b752e6175e, parent: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,105 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/6ce6d57f2d7946c2882e1228e898221c for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,126 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.splits/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-4de66ec55ffd49619300d5454630614b for child: 606b11da264cede32c10b7b752e6175e, parent: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,127 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/4de66ec55ffd49619300d5454630614b for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,139 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/.splits/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-41d58eb28fd140e2bbfa14d8ad1a463f for child: 606b11da264cede32c10b7b752e6175e, parent: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,140 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/41d58eb28fd140e2bbfa14d8ad1a463f for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,175 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,183 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,187 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,187 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,189 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,191 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,191 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,192 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933 for region: b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:40,192 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region b5548b9ced331770542633d75bc36e0d Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-13 12:15:40,241 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:15:40,243 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:15:40,246 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676290540246"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676290540246"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676290540246"}]},"ts":"1676290540246"} 2023-02-13 12:15:40,247 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676290540246"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290540246"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290540246"}]},"ts":"1676290540246"} 2023-02-13 12:15:40,247 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676290540246"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290540246"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290540246"}]},"ts":"1676290540246"} 2023-02-13 12:15:40,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6479942123147c70630ae414ae269a27, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=606b11da264cede32c10b7b752e6175e, ASSIGN}] 2023-02-13 12:15:40,291 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=606b11da264cede32c10b7b752e6175e, ASSIGN 2023-02-13 12:15:40,292 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=606b11da264cede32c10b7b752e6175e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:15:40,293 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6479942123147c70630ae414ae269a27, ASSIGN 2023-02-13 12:15:40,294 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6479942123147c70630ae414ae269a27, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:15:40,443 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 12:15:40,446 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=6479942123147c70630ae414ae269a27, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:40,447 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676290540446"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290540446"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290540446"}]},"ts":"1676290540446"} 2023-02-13 12:15:40,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 6479942123147c70630ae414ae269a27, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:15:40,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=606b11da264cede32c10b7b752e6175e, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:40,459 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676290540458"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290540458"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290540458"}]},"ts":"1676290540458"} 2023-02-13 12:15:40,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 606b11da264cede32c10b7b752e6175e, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:15:40,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:40,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 606b11da264cede32c10b7b752e6175e, NAME => 'testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.', STARTKEY => '26', ENDKEY => ''} 2023-02-13 12:15:40,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:40,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:40,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:40,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:40,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:40,617 INFO [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:40,619 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1 2023-02-13 12:15:40,619 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1 2023-02-13 12:15:40,619 INFO [StoreOpener-606b11da264cede32c10b7b752e6175e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 606b11da264cede32c10b7b752e6175e columnFamilyName cf1 2023-02-13 12:15:40,658 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72-top 2023-02-13 12:15:40,664 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933-top 2023-02-13 12:15:40,673 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d-top 2023-02-13 12:15:40,680 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561-top 2023-02-13 12:15:40,688 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-4de66ec55ffd49619300d5454630614b 2023-02-13 12:15:40,695 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-98d7b6d98fae444f9feca26096818335 2023-02-13 12:15:40,701 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-d27402b5359b4f6e8f249e7752df33b9 2023-02-13 12:15:40,702 INFO [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(310): Store=606b11da264cede32c10b7b752e6175e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:40,702 INFO [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:40,704 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2 2023-02-13 12:15:40,704 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2 2023-02-13 12:15:40,705 INFO [StoreOpener-606b11da264cede32c10b7b752e6175e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 606b11da264cede32c10b7b752e6175e columnFamilyName cf2 2023-02-13 12:15:40,723 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22-top 2023-02-13 12:15:40,729 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02-top 2023-02-13 12:15:40,735 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b-top 2023-02-13 12:15:40,743 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0-top 2023-02-13 12:15:40,749 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-3d27afb320c94d30a5fd4eae210a8b3b 2023-02-13 12:15:40,755 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-41d58eb28fd140e2bbfa14d8ad1a463f 2023-02-13 12:15:40,761 DEBUG [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-6ce6d57f2d7946c2882e1228e898221c 2023-02-13 12:15:40,761 INFO [StoreOpener-606b11da264cede32c10b7b752e6175e-1] regionserver.HStore(310): Store=606b11da264cede32c10b7b752e6175e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:40,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:40,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:40,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:15:40,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:40,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 606b11da264cede32c10b7b752e6175e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70539477, jitterRate=0.05112011730670929}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:15:40,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 606b11da264cede32c10b7b752e6175e: Running coprocessor pre-open hook at 1676290540614Writing region info on filesystem at 1676290540614Initializing all the Stores at 1676290540616 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290540617 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290540617Cleaning up temporary data from old regions at 1676290540765 (+148 ms)Cleaning up detritus from prior splits at 1676290540766 (+1 ms)Running coprocessor post-open hooks at 1676290540771 (+5 ms)Region opened successfully at 1676290540771 2023-02-13 12:15:40,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e., pid=18, masterSystemTime=1676290540607 2023-02-13 12:15:40,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:40,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:15:40,775 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:15:40,775 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:15:40,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:40,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:40,778 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:40,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:40,778 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1912): 606b11da264cede32c10b7b752e6175e/cf2 is initiating minor compaction (all files) 2023-02-13 12:15:40,778 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:40,778 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1912): 606b11da264cede32c10b7b752e6175e/cf1 is initiating minor compaction (all files) 2023-02-13 12:15:40,778 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 606b11da264cede32c10b7b752e6175e/cf2 in testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:40,778 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 606b11da264cede32c10b7b752e6175e/cf1 in testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:40,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6479942123147c70630ae414ae269a27, NAME => 'testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.', STARTKEY => '', ENDKEY => '26'} 2023-02-13 12:15:40,779 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=606b11da264cede32c10b7b752e6175e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:40,779 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-3d27afb320c94d30a5fd4eae210a8b3b, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-6ce6d57f2d7946c2882e1228e898221c, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-41d58eb28fd140e2bbfa14d8ad1a463f] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/.tmp, totalSize=34.9 K 2023-02-13 12:15:40,779 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-98d7b6d98fae444f9feca26096818335, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-4de66ec55ffd49619300d5454630614b, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-d27402b5359b4f6e8f249e7752df33b9] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/.tmp, totalSize=34.9 K 2023-02-13 12:15:40,779 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676290540779"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290540779"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290540779"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290540779"}]},"ts":"1676290540779"} 2023-02-13 12:15:40,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:40,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 6479942123147c70630ae414ae269a27 2023-02-13 12:15:40,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:40,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 6479942123147c70630ae414ae269a27 2023-02-13 12:15:40,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 6479942123147c70630ae414ae269a27 2023-02-13 12:15:40,782 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676290537530 2023-02-13 12:15:40,783 INFO [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6479942123147c70630ae414ae269a27 2023-02-13 12:15:40,784 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676290538009 2023-02-13 12:15:40,785 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676290538347 2023-02-13 12:15:40,786 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1 2023-02-13 12:15:40,786 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1 2023-02-13 12:15:40,786 INFO [StoreOpener-6479942123147c70630ae414ae269a27-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6479942123147c70630ae414ae269a27 columnFamilyName cf1 2023-02-13 12:15:40,786 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676290537530 2023-02-13 12:15:40,786 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676290538672 2023-02-13 12:15:40,790 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-3d27afb320c94d30a5fd4eae210a8b3b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290538991 2023-02-13 12:15:40,790 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676290538009 2023-02-13 12:15:40,791 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-6ce6d57f2d7946c2882e1228e898221c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290539330 2023-02-13 12:15:40,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-13 12:15:40,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 606b11da264cede32c10b7b752e6175e, server=jenkins-hbase16.apache.org,33055,1676290533708 in 324 msec 2023-02-13 12:15:40,791 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676290538347 2023-02-13 12:15:40,792 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-41d58eb28fd140e2bbfa14d8ad1a463f, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290539641 2023-02-13 12:15:40,792 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676290538672 2023-02-13 12:15:40,794 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-98d7b6d98fae444f9feca26096818335, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290538991 2023-02-13 12:15:40,795 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-4de66ec55ffd49619300d5454630614b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290539330 2023-02-13 12:15:40,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=606b11da264cede32c10b7b752e6175e, ASSIGN in 501 msec 2023-02-13 12:15:40,796 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-d27402b5359b4f6e8f249e7752df33b9, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290539641 2023-02-13 12:15:40,818 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72-bottom 2023-02-13 12:15:40,828 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] throttle.PressureAwareThroughputController(145): 606b11da264cede32c10b7b752e6175e#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:15:40,828 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933-bottom 2023-02-13 12:15:40,833 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] throttle.PressureAwareThroughputController(145): 606b11da264cede32c10b7b752e6175e#cf1#compaction#15 average throughput is 0.32 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:15:40,836 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d-bottom 2023-02-13 12:15:40,841 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561-bottom 2023-02-13 12:15:40,841 INFO [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(310): Store=6479942123147c70630ae414ae269a27/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:40,841 INFO [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6479942123147c70630ae414ae269a27 2023-02-13 12:15:40,843 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2 2023-02-13 12:15:40,843 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2 2023-02-13 12:15:40,843 INFO [StoreOpener-6479942123147c70630ae414ae269a27-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6479942123147c70630ae414ae269a27 columnFamilyName cf2 2023-02-13 12:15:40,851 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/.tmp/cf2/e8aaa3520b634e35887a4ffef6091eb0 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/e8aaa3520b634e35887a4ffef6091eb0 2023-02-13 12:15:40,855 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/.tmp/cf1/fc84d5822c2948a5b51ef34e4dab957e as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/fc84d5822c2948a5b51ef34e4dab957e 2023-02-13 12:15:40,859 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22-bottom 2023-02-13 12:15:40,867 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02-bottom 2023-02-13 12:15:40,869 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 606b11da264cede32c10b7b752e6175e/cf2 of 606b11da264cede32c10b7b752e6175e into e8aaa3520b634e35887a4ffef6091eb0(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:15:40,869 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 606b11da264cede32c10b7b752e6175e/cf1 of 606b11da264cede32c10b7b752e6175e into fc84d5822c2948a5b51ef34e4dab957e(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:15:40,869 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 606b11da264cede32c10b7b752e6175e: 2023-02-13 12:15:40,869 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 606b11da264cede32c10b7b752e6175e: 2023-02-13 12:15:40,869 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e., storeName=606b11da264cede32c10b7b752e6175e/cf1, priority=9, startTime=1676290540774; duration=0sec 2023-02-13 12:15:40,869 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e., storeName=606b11da264cede32c10b7b752e6175e/cf2, priority=9, startTime=1676290540775; duration=0sec 2023-02-13 12:15:40,870 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:40,870 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:40,873 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b-bottom 2023-02-13 12:15:40,877 DEBUG [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0-bottom 2023-02-13 12:15:40,877 INFO [StoreOpener-6479942123147c70630ae414ae269a27-1] regionserver.HStore(310): Store=6479942123147c70630ae414ae269a27/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:40,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27 2023-02-13 12:15:40,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27 2023-02-13 12:15:40,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:15:40,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 6479942123147c70630ae414ae269a27 2023-02-13 12:15:40,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 6479942123147c70630ae414ae269a27; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65449324, jitterRate=-0.02472907304763794}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:15:40,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 6479942123147c70630ae414ae269a27: Running coprocessor pre-open hook at 1676290540781Writing region info on filesystem at 1676290540781Initializing all the Stores at 1676290540783 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290540783Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290540783Cleaning up temporary data from old regions at 1676290540880 (+97 ms)Cleaning up detritus from prior splits at 1676290540880Running coprocessor post-open hooks at 1676290540883 (+3 ms)Region opened successfully at 1676290540883 2023-02-13 12:15:40,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27., pid=17, masterSystemTime=1676290540607 2023-02-13 12:15:40,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:15:40,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:15:40,885 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 12:15:40,885 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 12:15:40,885 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:40,885 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:40,885 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1912): 6479942123147c70630ae414ae269a27/cf2 is initiating minor compaction (all files) 2023-02-13 12:15:40,885 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1912): 6479942123147c70630ae414ae269a27/cf1 is initiating minor compaction (all files) 2023-02-13 12:15:40,885 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6479942123147c70630ae414ae269a27/cf2 in testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:40,885 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6479942123147c70630ae414ae269a27/cf1 in testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:40,885 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d-bottom] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/.tmp, totalSize=19.9 K 2023-02-13 12:15:40,885 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02-bottom] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/.tmp, totalSize=19.9 K 2023-02-13 12:15:40,886 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290537530 2023-02-13 12:15:40,886 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290537530 2023-02-13 12:15:40,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:40,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:40,887 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290538009 2023-02-13 12:15:40,887 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290538009 2023-02-13 12:15:40,888 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=6479942123147c70630ae414ae269a27, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:40,888 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676290540888"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290540888"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290540888"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290540888"}]},"ts":"1676290540888"} 2023-02-13 12:15:40,889 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290538347 2023-02-13 12:15:40,889 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290538347 2023-02-13 12:15:40,889 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290538672 2023-02-13 12:15:40,889 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290538672 2023-02-13 12:15:40,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-13 12:15:40,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 6479942123147c70630ae414ae269a27, server=jenkins-hbase16.apache.org,33055,1676290533708 in 438 msec 2023-02-13 12:15:40,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-13 12:15:40,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6479942123147c70630ae414ae269a27, ASSIGN in 604 msec 2023-02-13 12:15:40,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e in 1.2390 sec 2023-02-13 12:15:40,906 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6479942123147c70630ae414ae269a27#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:15:40,906 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] throttle.PressureAwareThroughputController(145): 6479942123147c70630ae414ae269a27#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:15:40,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33055] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 188.40.62.62:45960 deadline: 1676290600917, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d. is not online on jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:40,923 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/.tmp/cf2/40c693aa66754440af9660b57e21726f as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/40c693aa66754440af9660b57e21726f 2023-02-13 12:15:40,923 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/.tmp/cf1/9896cd0fe0a94df99d8495bc34e387bb as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/9896cd0fe0a94df99d8495bc34e387bb 2023-02-13 12:15:40,930 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 6479942123147c70630ae414ae269a27/cf2 of 6479942123147c70630ae414ae269a27 into 40c693aa66754440af9660b57e21726f(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:15:40,930 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 6479942123147c70630ae414ae269a27/cf1 of 6479942123147c70630ae414ae269a27 into 9896cd0fe0a94df99d8495bc34e387bb(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:15:40,930 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6479942123147c70630ae414ae269a27: 2023-02-13 12:15:40,930 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6479942123147c70630ae414ae269a27: 2023-02-13 12:15:40,930 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27., storeName=6479942123147c70630ae414ae269a27/cf2, priority=12, startTime=1676290540884; duration=0sec 2023-02-13 12:15:40,930 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27., storeName=6479942123147c70630ae414ae269a27/cf1, priority=12, startTime=1676290540884; duration=0sec 2023-02-13 12:15:40,930 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:40,930 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:41,055 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:15:41,063 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 12:15:41,085 INFO [Listener at localhost.localdomain/46087] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 12:15:41,089 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 12:15:41,123 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:15:41,132 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-13 12:15:41,138 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitTableRegionUnevenDaughter 2023-02-13 12:15:41,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:41,150 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290541150"}]},"ts":"1676290541150"} 2023-02-13 12:15:41,150 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 12:15:41,151 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 12:15:41,152 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-13 12:15:41,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 12:15:41,190 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-13 12:15:41,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6479942123147c70630ae414ae269a27, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=606b11da264cede32c10b7b752e6175e, UNASSIGN}] 2023-02-13 12:15:41,193 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=606b11da264cede32c10b7b752e6175e, UNASSIGN 2023-02-13 12:15:41,194 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=606b11da264cede32c10b7b752e6175e, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:41,194 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676290541194"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290541194"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290541194"}]},"ts":"1676290541194"} 2023-02-13 12:15:41,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 606b11da264cede32c10b7b752e6175e, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:15:41,199 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6479942123147c70630ae414ae269a27, UNASSIGN 2023-02-13 12:15:41,200 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=6479942123147c70630ae414ae269a27, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:41,201 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676290541200"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290541200"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290541200"}]},"ts":"1676290541200"} 2023-02-13 12:15:41,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 6479942123147c70630ae414ae269a27, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:15:41,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 12:15:41,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:41,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 606b11da264cede32c10b7b752e6175e, disabling compactions & flushes 2023-02-13 12:15:41,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:41,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:41,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. after waiting 0 ms 2023-02-13 12:15:41,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:41,369 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-98d7b6d98fae444f9feca26096818335, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-4de66ec55ffd49619300d5454630614b, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-d27402b5359b4f6e8f249e7752df33b9] to archive 2023-02-13 12:15:41,371 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:15:41,376 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,377 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,379 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,380 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,381 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-98d7b6d98fae444f9feca26096818335 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-98d7b6d98fae444f9feca26096818335 2023-02-13 12:15:41,383 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-4de66ec55ffd49619300d5454630614b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-4de66ec55ffd49619300d5454630614b 2023-02-13 12:15:41,384 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-d27402b5359b4f6e8f249e7752df33b9 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-d27402b5359b4f6e8f249e7752df33b9 2023-02-13 12:15:41,403 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-3d27afb320c94d30a5fd4eae210a8b3b, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-6ce6d57f2d7946c2882e1228e898221c, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-41d58eb28fd140e2bbfa14d8ad1a463f] to archive 2023-02-13 12:15:41,404 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:15:41,406 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,407 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,409 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,410 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,412 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-3d27afb320c94d30a5fd4eae210a8b3b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-3d27afb320c94d30a5fd4eae210a8b3b 2023-02-13 12:15:41,414 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-6ce6d57f2d7946c2882e1228e898221c to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-6ce6d57f2d7946c2882e1228e898221c 2023-02-13 12:15:41,416 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-41d58eb28fd140e2bbfa14d8ad1a463f to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/testSplitTableRegionUnevenDaughter=b5548b9ced331770542633d75bc36e0d-41d58eb28fd140e2bbfa14d8ad1a463f 2023-02-13 12:15:41,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 12:15:41,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:15:41,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e. 2023-02-13 12:15:41,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 606b11da264cede32c10b7b752e6175e: Waiting for close lock at 1676290541349Running coprocessor pre-close hooks at 1676290541349Disabling compacts and flushes for region at 1676290541349Disabling writes for close at 1676290541350 (+1 ms)Writing region close event to WAL at 1676290541417 (+67 ms)Running coprocessor post-close hooks at 1676290541422 (+5 ms)Closed at 1676290541423 (+1 ms) 2023-02-13 12:15:41,426 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:41,426 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 6479942123147c70630ae414ae269a27 2023-02-13 12:15:41,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 6479942123147c70630ae414ae269a27, disabling compactions & flushes 2023-02-13 12:15:41,426 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:41,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:41,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. after waiting 0 ms 2023-02-13 12:15:41,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:41,427 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=606b11da264cede32c10b7b752e6175e, regionState=CLOSED 2023-02-13 12:15:41,427 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676290541427"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290541427"}]},"ts":"1676290541427"} 2023-02-13 12:15:41,427 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d-bottom] to archive 2023-02-13 12:15:41,428 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:15:41,431 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/dd75ef112ba9482782912780eb772561.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-13 12:15:41,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 606b11da264cede32c10b7b752e6175e, server=jenkins-hbase16.apache.org,33055,1676290533708 in 233 msec 2023-02-13 12:15:41,433 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/5762a6a063604969948f9629eb8ba933.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,435 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/417a65eaac054a87bde5350a6a03da72.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=606b11da264cede32c10b7b752e6175e, UNASSIGN in 241 msec 2023-02-13 12:15:41,436 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/b1d1b836d4b046d0a377fc1fbfccea8d.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,438 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02-bottom] to archive 2023-02-13 12:15:41,439 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:15:41,441 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/08b7e875196a481b8275cfed87a2fb22.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,443 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/dd23553b9a3644249a7673f027fac1d0.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,445 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/c068274c957b4a6dafcc96e6992d674b.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,446 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/8d338657e44f4eec946680459b7a4e02.b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 12:15:41,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:15:41,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27. 2023-02-13 12:15:41,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 6479942123147c70630ae414ae269a27: Waiting for close lock at 1676290541426Running coprocessor pre-close hooks at 1676290541426Disabling compacts and flushes for region at 1676290541426Disabling writes for close at 1676290541426Writing region close event to WAL at 1676290541448 (+22 ms)Running coprocessor post-close hooks at 1676290541452 (+4 ms)Closed at 1676290541454 (+2 ms) 2023-02-13 12:15:41,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 6479942123147c70630ae414ae269a27 2023-02-13 12:15:41,457 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=6479942123147c70630ae414ae269a27, regionState=CLOSED 2023-02-13 12:15:41,457 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676290541457"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290541457"}]},"ts":"1676290541457"} 2023-02-13 12:15:41,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-13 12:15:41,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 12:15:41,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 6479942123147c70630ae414ae269a27, server=jenkins-hbase16.apache.org,33055,1676290533708 in 258 msec 2023-02-13 12:15:41,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-13 12:15:41,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6479942123147c70630ae414ae269a27, UNASSIGN in 271 msec 2023-02-13 12:15:41,467 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290541467"}]},"ts":"1676290541467"} 2023-02-13 12:15:41,469 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-13 12:15:41,476 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-13 12:15:41,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 337 msec 2023-02-13 12:15:41,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 12:15:41,765 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-13 12:15:41,769 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitTableRegionUnevenDaughter 2023-02-13 12:15:41,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:41,781 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:41,783 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:41,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 12:15:41,792 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,792 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27 2023-02-13 12:15:41,792 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:41,796 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/recovered.edits] 2023-02-13 12:15:41,796 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/recovered.edits] 2023-02-13 12:15:41,796 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/recovered.edits] 2023-02-13 12:15:41,800 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/9896cd0fe0a94df99d8495bc34e387bb to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf1/9896cd0fe0a94df99d8495bc34e387bb 2023-02-13 12:15:41,800 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/fc84d5822c2948a5b51ef34e4dab957e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf1/fc84d5822c2948a5b51ef34e4dab957e 2023-02-13 12:15:41,801 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/417a65eaac054a87bde5350a6a03da72 2023-02-13 12:15:41,803 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/40c693aa66754440af9660b57e21726f to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/cf2/40c693aa66754440af9660b57e21726f 2023-02-13 12:15:41,803 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/e8aaa3520b634e35887a4ffef6091eb0 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/cf2/e8aaa3520b634e35887a4ffef6091eb0 2023-02-13 12:15:41,804 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/4de66ec55ffd49619300d5454630614b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/4de66ec55ffd49619300d5454630614b 2023-02-13 12:15:41,805 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/5762a6a063604969948f9629eb8ba933 2023-02-13 12:15:41,806 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/recovered.edits/90.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27/recovered.edits/90.seqid 2023-02-13 12:15:41,807 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/recovered.edits/90.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e/recovered.edits/90.seqid 2023-02-13 12:15:41,807 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/98d7b6d98fae444f9feca26096818335 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/98d7b6d98fae444f9feca26096818335 2023-02-13 12:15:41,807 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/6479942123147c70630ae414ae269a27 2023-02-13 12:15:41,808 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:41,809 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/b1d1b836d4b046d0a377fc1fbfccea8d 2023-02-13 12:15:41,810 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/d27402b5359b4f6e8f249e7752df33b9 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/d27402b5359b4f6e8f249e7752df33b9 2023-02-13 12:15:41,811 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf1/dd75ef112ba9482782912780eb772561 2023-02-13 12:15:41,815 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/08b7e875196a481b8275cfed87a2fb22 2023-02-13 12:15:41,816 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/3d27afb320c94d30a5fd4eae210a8b3b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/3d27afb320c94d30a5fd4eae210a8b3b 2023-02-13 12:15:41,818 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/41d58eb28fd140e2bbfa14d8ad1a463f to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/41d58eb28fd140e2bbfa14d8ad1a463f 2023-02-13 12:15:41,819 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/6ce6d57f2d7946c2882e1228e898221c to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/6ce6d57f2d7946c2882e1228e898221c 2023-02-13 12:15:41,820 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/8d338657e44f4eec946680459b7a4e02 2023-02-13 12:15:41,822 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/c068274c957b4a6dafcc96e6992d674b 2023-02-13 12:15:41,823 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/cf2/dd23553b9a3644249a7673f027fac1d0 2023-02-13 12:15:41,826 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/recovered.edits/85.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d/recovered.edits/85.seqid 2023-02-13 12:15:41,826 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionUnevenDaughter/b5548b9ced331770542633d75bc36e0d 2023-02-13 12:15:41,826 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 12:15:41,830 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:41,854 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-13 12:15:41,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 12:15:41,903 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-13 12:15:41,905 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:41,905 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-13 12:15:41,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290541905"}]},"ts":"9223372036854775807"} 2023-02-13 12:15:41,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290541905"}]},"ts":"9223372036854775807"} 2023-02-13 12:15:41,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290541905"}]},"ts":"9223372036854775807"} 2023-02-13 12:15:41,912 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 12:15:41,912 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b5548b9ced331770542633d75bc36e0d, NAME => 'testSplitTableRegionUnevenDaughter,,1676290536765.b5548b9ced331770542633d75bc36e0d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6479942123147c70630ae414ae269a27, NAME => 'testSplitTableRegionUnevenDaughter,,1676290539659.6479942123147c70630ae414ae269a27.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 606b11da264cede32c10b7b752e6175e, NAME => 'testSplitTableRegionUnevenDaughter,26,1676290539659.606b11da264cede32c10b7b752e6175e.', STARTKEY => '26', ENDKEY => ''}] 2023-02-13 12:15:41,912 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-13 12:15:41,913 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676290541912"}]},"ts":"9223372036854775807"} 2023-02-13 12:15:41,915 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-13 12:15:41,927 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:41,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 157 msec 2023-02-13 12:15:42,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 12:15:42,088 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-13 12:15:42,106 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=394 (was 361) Potentially hanging thread: member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:49346 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3fbde3a3-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:42790 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:34056 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:48332 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:49306 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:48436 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:48322 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3fbde3a3-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:48342 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:42816 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:48336 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3fbde3a3-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3fbde3a3-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:49330 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:34080 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase16:33055-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:34042 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3fbde3a3-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:34106 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:48482 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:49444 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=835 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=247 (was 269), ProcessCount=167 (was 167), AvailableMemoryMB=6254 (was 6082) - AvailableMemoryMB LEAK? - 2023-02-13 12:15:42,121 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=394, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=247, ProcessCount=167, AvailableMemoryMB=6251 2023-02-13 12:15:42,121 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:15:42,156 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 12:15:42,156 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 12:15:42,158 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:42,160 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:15:42,161 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:15:42,163 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:42,164 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353 empty. 2023-02-13 12:15:42,164 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:42,164 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 12:15:42,176 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-13 12:15:42,177 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3bb9f1de2ea5395a2121754580ffb353, NAME => 'testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:15:42,197 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:42,197 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 3bb9f1de2ea5395a2121754580ffb353, disabling compactions & flushes 2023-02-13 12:15:42,197 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:42,198 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:42,198 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. after waiting 0 ms 2023-02-13 12:15:42,198 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:42,198 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:42,198 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 3bb9f1de2ea5395a2121754580ffb353: Waiting for close lock at 1676290542197Disabling compacts and flushes for region at 1676290542197Disabling writes for close at 1676290542198 (+1 ms)Writing region close event to WAL at 1676290542198Closed at 1676290542198 2023-02-13 12:15:42,201 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:15:42,202 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290542202"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290542202"}]},"ts":"1676290542202"} 2023-02-13 12:15:42,206 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:15:42,207 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:15:42,208 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290542207"}]},"ts":"1676290542207"} 2023-02-13 12:15:42,210 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-13 12:15:42,231 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:15:42,233 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:15:42,233 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:15:42,233 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:15:42,233 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:15:42,234 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, ASSIGN}] 2023-02-13 12:15:42,237 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, ASSIGN 2023-02-13 12:15:42,238 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,45345,1676290533799; forceNewPlan=false, retain=false 2023-02-13 12:15:42,388 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:15:42,389 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=3bb9f1de2ea5395a2121754580ffb353, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:42,389 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290542389"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290542389"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290542389"}]},"ts":"1676290542389"} 2023-02-13 12:15:42,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:15:42,545 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:42,545 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 12:15:42,546 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:50690, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 12:15:42,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:42,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3bb9f1de2ea5395a2121754580ffb353, NAME => 'testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:15:42,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:42,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:42,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:42,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:42,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:42,553 INFO [StoreOpener-3bb9f1de2ea5395a2121754580ffb353-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:42,555 DEBUG [StoreOpener-3bb9f1de2ea5395a2121754580ffb353-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1 2023-02-13 12:15:42,555 DEBUG [StoreOpener-3bb9f1de2ea5395a2121754580ffb353-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1 2023-02-13 12:15:42,555 INFO [StoreOpener-3bb9f1de2ea5395a2121754580ffb353-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bb9f1de2ea5395a2121754580ffb353 columnFamilyName cf1 2023-02-13 12:15:42,556 INFO [StoreOpener-3bb9f1de2ea5395a2121754580ffb353-1] regionserver.HStore(310): Store=3bb9f1de2ea5395a2121754580ffb353/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:42,556 INFO [StoreOpener-3bb9f1de2ea5395a2121754580ffb353-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:42,558 DEBUG [StoreOpener-3bb9f1de2ea5395a2121754580ffb353-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2 2023-02-13 12:15:42,558 DEBUG [StoreOpener-3bb9f1de2ea5395a2121754580ffb353-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2 2023-02-13 12:15:42,559 INFO [StoreOpener-3bb9f1de2ea5395a2121754580ffb353-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bb9f1de2ea5395a2121754580ffb353 columnFamilyName cf2 2023-02-13 12:15:42,560 INFO [StoreOpener-3bb9f1de2ea5395a2121754580ffb353-1] regionserver.HStore(310): Store=3bb9f1de2ea5395a2121754580ffb353/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:42,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:42,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:42,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:15:42,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:42,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:15:42,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 3bb9f1de2ea5395a2121754580ffb353; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60190031, jitterRate=-0.10309864580631256}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:15:42,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 3bb9f1de2ea5395a2121754580ffb353: Running coprocessor pre-open hook at 1676290542552Writing region info on filesystem at 1676290542552Initializing all the Stores at 1676290542553 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290542553Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290542553Cleaning up temporary data from old regions at 1676290542562 (+9 ms)Cleaning up detritus from prior splits at 1676290542563 (+1 ms)Running coprocessor post-open hooks at 1676290542585 (+22 ms)Region opened successfully at 1676290542585 2023-02-13 12:15:42,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353., pid=27, masterSystemTime=1676290542545 2023-02-13 12:15:42,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:42,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:42,598 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=3bb9f1de2ea5395a2121754580ffb353, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:42,598 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290542597"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290542597"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290542597"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290542597"}]},"ts":"1676290542597"} 2023-02-13 12:15:42,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-13 12:15:42,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 in 209 msec 2023-02-13 12:15:42,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-13 12:15:42,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, ASSIGN in 370 msec 2023-02-13 12:15:42,607 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:15:42,607 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290542607"}]},"ts":"1676290542607"} 2023-02-13 12:15:42,609 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-13 12:15:42,669 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:15:42,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 512 msec 2023-02-13 12:15:42,791 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 12:15:42,791 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 12:15:42,792 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:15:42,792 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-13 12:15:42,912 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:15:42,915 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:50692, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:15:42,924 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:42,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 12:15:42,926 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 12:15:42,926 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:42,927 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 12:15:42,927 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 12:15:42,928 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:42,928 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:42,934 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:42,934 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:42,934 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:42,934 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:42,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:42,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:42,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:42,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:42,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:42,935 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:42,935 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:42,935 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:42,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:42,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:42,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:42,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:42,936 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:42,936 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:42,936 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:42,936 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:42,937 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:42,938 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:42,938 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:42,938 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:42,938 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:42,938 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:42,938 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:42,939 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:42,938 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:42,939 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:42,939 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:42,939 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. started... 2023-02-13 12:15:42,939 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:42,939 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:42,940 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bb9f1de2ea5395a2121754580ffb353 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 12:15:42,950 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:42,955 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/a7306eb0442640abaf8bfc64a0e46111 2023-02-13 12:15:42,959 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:42,959 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:42,959 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:42,959 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:42,959 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:42,986 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/0759008837e747bb91986c986dd15557 2023-02-13 12:15:42,993 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/a7306eb0442640abaf8bfc64a0e46111 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111 2023-02-13 12:15:42,999 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:15:43,000 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/0759008837e747bb91986c986dd15557 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557 2023-02-13 12:15:43,008 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:15:43,010 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3bb9f1de2ea5395a2121754580ffb353 in 70ms, sequenceid=6, compaction requested=false 2023-02-13 12:15:43,010 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-13 12:15:43,011 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bb9f1de2ea5395a2121754580ffb353: 2023-02-13 12:15:43,011 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:43,011 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:43,011 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:43,011 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,011 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:43,011 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:43,017 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,017 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:43,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:43,018 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,018 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:43,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:43,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:43,020 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 12:15:43,020 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 12:15:43,020 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4933efa8[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:43,020 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,025 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,026 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,026 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,026 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:43,026 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,026 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:43,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,026 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:43,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,026 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:43,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,026 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:43,026 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:43,026 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:43,026 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:43,026 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:43,026 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:43,034 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:43,034 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,034 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:43,042 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:43,042 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,042 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:43,042 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,043 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:43,042 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,043 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:43,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:43,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:15:43,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 12:15:43,048 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 12:15:43,048 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:43,048 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:43,048 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 12:15:43,048 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:43,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,060 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:43,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:43,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,060 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:43,061 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:43,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,064 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:43,064 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,065 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,076 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 12:15:43,076 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:43,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,076 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,077 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:43,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,076 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,077 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:43,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,077 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,078 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,079 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,079 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,079 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,079 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,178 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:43,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:43,198 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:43,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:43,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 12:15:43,201 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 12:15:43,201 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,201 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 12:15:43,201 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 12:15:43,202 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,202 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,209 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,209 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,209 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,209 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,209 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,209 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,210 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:43,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:43,212 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:43,212 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:43,212 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,212 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:43,213 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:43,213 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:43,213 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,212 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:43,213 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:43,213 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:43,213 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:43,213 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:43,214 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. started... 2023-02-13 12:15:43,213 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:43,214 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:43,213 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,214 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:43,214 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:43,214 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bb9f1de2ea5395a2121754580ffb353 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:43,214 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:43,214 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:43,214 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:43,223 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,230 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/47ab6e8111334a9690a1befe29f29813 2023-02-13 12:15:43,231 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,231 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:43,231 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,231 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,231 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:43,246 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/33c1bc2d753b44499adbadcc8bd66108 2023-02-13 12:15:43,252 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/47ab6e8111334a9690a1befe29f29813 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813 2023-02-13 12:15:43,258 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:15:43,259 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/33c1bc2d753b44499adbadcc8bd66108 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108 2023-02-13 12:15:43,265 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:15:43,266 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3bb9f1de2ea5395a2121754580ffb353 in 52ms, sequenceid=19, compaction requested=false 2023-02-13 12:15:43,266 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bb9f1de2ea5395a2121754580ffb353: 2023-02-13 12:15:43,266 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:43,266 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:43,266 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:43,266 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,266 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:43,266 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:43,275 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,276 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:43,276 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,276 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:43,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 12:15:43,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5fad25ae[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:43,278 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 12:15:43,278 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,284 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,284 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,284 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,284 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,284 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:43,284 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,284 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:43,284 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:43,284 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,284 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:43,284 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:43,284 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:43,285 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:43,285 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:43,341 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:43,341 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,341 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:43,359 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:43,359 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,359 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:43,359 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,359 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,359 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:43,360 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:43,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:43,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:43,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:15:43,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 12:15:43,364 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 12:15:43,365 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:43,365 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:43,365 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 12:15:43,365 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:43,401 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:43,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:43,401 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,401 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,401 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:43,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,402 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:43,402 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,404 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:43,404 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,404 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,442 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,442 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,442 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,443 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 12:15:43,443 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:43,442 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,443 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,443 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,443 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,443 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,443 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,443 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:43,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,443 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,443 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:43,443 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,443 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,443 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,444 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,444 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,444 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,444 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,444 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,544 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:43,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:43,596 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:43,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:43,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 12:15:43,599 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 12:15:43,599 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,600 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 12:15:43,601 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 12:15:43,601 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,601 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,614 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,614 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,615 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,615 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,617 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,618 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:43,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:43,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:43,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,619 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:43,619 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:43,619 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:43,619 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:43,619 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:43,619 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:43,619 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:43,619 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:43,619 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,619 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,620 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:43,620 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:43,620 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:43,620 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:43,620 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:43,620 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:43,620 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:43,620 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:43,621 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:43,621 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,621 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:43,621 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:43,621 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:43,621 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:43,622 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. started... 2023-02-13 12:15:43,622 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bb9f1de2ea5395a2121754580ffb353 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:43,631 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,639 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,639 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,640 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:43,640 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,640 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:43,642 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/c29367d79c0a4e1d98a2733e505eb6d2 2023-02-13 12:15:43,658 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/11e2cc96f05e422aa075e4fae58d4ac0 2023-02-13 12:15:43,664 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/c29367d79c0a4e1d98a2733e505eb6d2 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2 2023-02-13 12:15:43,670 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:15:43,671 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/11e2cc96f05e422aa075e4fae58d4ac0 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0 2023-02-13 12:15:43,676 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:15:43,677 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3bb9f1de2ea5395a2121754580ffb353 in 55ms, sequenceid=32, compaction requested=true 2023-02-13 12:15:43,677 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bb9f1de2ea5395a2121754580ffb353: 2023-02-13 12:15:43,677 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:43,677 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:43,677 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:43,678 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,678 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:43,678 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:43,692 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,692 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:43,693 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,693 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:43,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 12:15:43,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e8887f3[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:43,695 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 12:15:43,695 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,701 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,701 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,701 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,701 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,701 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:43,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,701 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:43,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,701 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:43,701 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:43,701 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:43,702 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:43,701 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:43,702 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:43,702 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:43,702 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:43,709 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:43,709 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,709 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:43,717 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:43,717 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,717 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:43,717 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,718 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:43,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:43,717 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:43,718 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:43,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:43,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:43,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:43,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:15:43,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 12:15:43,726 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 12:15:43,726 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:43,726 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:43,726 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 12:15:43,726 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:43,734 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,734 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,734 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,734 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,734 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,734 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,737 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:43,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:43,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:43,741 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,741 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:43,741 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:43,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,744 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,745 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,745 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,756 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,756 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,757 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,756 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,756 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,757 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,757 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 12:15:43,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:43,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,757 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,757 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:43,757 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:43,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,757 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,757 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:43,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,758 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,758 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:43,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,758 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,758 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,758 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,758 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,758 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,758 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:43,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:43,858 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:43,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:43,893 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:43,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:43,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 12:15:43,895 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 12:15:43,895 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,896 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 12:15:43,896 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 12:15:43,896 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,896 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,927 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:43,927 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,927 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:43,928 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:43,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:43,929 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:43,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:43,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:43,929 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:43,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:43,929 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:43,929 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:43,929 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:43,929 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:43,929 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:43,929 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,929 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,930 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:43,931 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:43,931 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:43,931 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:43,929 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:43,931 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:43,933 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:43,933 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:43,933 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:43,933 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. started... 2023-02-13 12:15:43,933 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:43,933 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:43,933 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bb9f1de2ea5395a2121754580ffb353 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:43,944 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/146d5b7be2d94cafb42bbd42cb0f65ec 2023-02-13 12:15:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,956 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,956 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:43,956 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,957 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,957 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:43,959 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/8ba6dd0a1459467fb94a960d2708ca2b 2023-02-13 12:15:43,965 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/146d5b7be2d94cafb42bbd42cb0f65ec as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec 2023-02-13 12:15:43,970 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:15:43,971 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/8ba6dd0a1459467fb94a960d2708ca2b as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b 2023-02-13 12:15:43,978 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:15:43,979 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3bb9f1de2ea5395a2121754580ffb353 in 46ms, sequenceid=45, compaction requested=true 2023-02-13 12:15:43,980 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bb9f1de2ea5395a2121754580ffb353: 2023-02-13 12:15:43,980 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:43,980 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:43,980 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:43,980 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,980 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:43,980 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:43,989 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,989 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:43,990 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:43,990 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:43,992 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 12:15:43,992 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@539f3f7[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:43,992 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 12:15:43,992 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,001 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,001 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,001 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,001 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:44,001 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:44,001 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,001 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:44,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,001 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:44,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,001 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:44,001 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:44,002 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:44,001 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:44,002 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:44,002 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:44,009 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:44,009 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,009 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:44,017 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:44,017 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,017 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:44,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:44,017 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:44,026 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:44,026 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,026 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:15:44,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 12:15:44,028 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 12:15:44,028 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:44,028 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:44,028 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 12:15:44,028 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:44,034 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,034 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,034 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,034 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,034 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,034 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,034 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,034 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:44,035 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:44,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,036 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:44,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:44,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,038 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,038 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,038 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,051 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,051 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,051 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,051 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,051 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,051 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,051 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,051 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,051 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 12:15:44,051 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:44,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,051 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,052 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:44,051 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,052 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:44,052 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,052 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,052 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,052 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,052 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,052 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,052 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,052 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,152 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:44,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:44,175 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:44,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:44,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 12:15:44,177 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 12:15:44,177 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:44,177 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 12:15:44,177 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 12:15:44,177 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,177 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,216 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,216 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,216 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,216 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,217 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,217 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:44,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,218 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:44,218 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:44,218 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:44,219 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:44,219 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:44,219 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:44,219 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:44,219 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:44,219 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:44,219 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:44,219 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:44,219 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:44,219 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:44,219 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:44,219 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:44,219 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:44,220 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:44,220 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:44,219 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:44,220 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:44,220 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:44,220 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. started... 2023-02-13 12:15:44,220 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:44,220 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bb9f1de2ea5395a2121754580ffb353 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:44,235 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/5d8c1c02e42149039cb6615b911d54fa 2023-02-13 12:15:44,254 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/a70f882bb2ec496ead2f96d700399c24 2023-02-13 12:15:44,260 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/5d8c1c02e42149039cb6615b911d54fa as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa 2023-02-13 12:15:44,265 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:15:44,266 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/a70f882bb2ec496ead2f96d700399c24 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24 2023-02-13 12:15:44,271 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:15:44,272 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3bb9f1de2ea5395a2121754580ffb353 in 52ms, sequenceid=58, compaction requested=true 2023-02-13 12:15:44,273 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bb9f1de2ea5395a2121754580ffb353: 2023-02-13 12:15:44,273 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:44,273 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:44,273 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:44,273 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,273 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:44,273 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:44,276 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,284 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,284 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,284 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,284 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:44,284 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,284 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,284 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:44,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,285 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,285 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:44,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:44,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:44,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:44,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:44,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:44,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 12:15:44,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53210257[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:44,287 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 12:15:44,287 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,292 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,292 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,292 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,293 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:44,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,293 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,293 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:44,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,293 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:44,293 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:44,293 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:44,293 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:44,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,293 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:44,293 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:44,293 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:44,293 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:44,301 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:44,301 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,301 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:44,309 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,309 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:44,309 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,309 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:44,309 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:44,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:44,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:44,309 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,309 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:44,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:44,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:44,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:15:44,312 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 12:15:44,312 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 12:15:44,312 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:44,312 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:44,312 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 12:15:44,312 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:44,381 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,381 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,381 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,381 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,382 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:44,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:44,382 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,382 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,383 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:44,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,384 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,385 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:44,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,388 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:44,389 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,390 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,409 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,409 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 12:15:44,410 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:44,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,410 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,410 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:44,410 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:44,410 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,410 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,411 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,411 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,411 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,510 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:44,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:44,533 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:44,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:44,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 12:15:44,535 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 12:15:44,535 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:44,535 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 12:15:44,535 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 12:15:44,535 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,536 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,542 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,542 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,542 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,542 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,543 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,543 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:44,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 12:15:44,544 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:44,544 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:44,544 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:44,544 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:44,544 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:44,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:44,545 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:44,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:44,545 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:44,545 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:44,546 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. started... 2023-02-13 12:15:44,546 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bb9f1de2ea5395a2121754580ffb353 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:44,550 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,559 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,559 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,559 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:44,559 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,559 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:44,567 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/026bf871cef740b09aecf686e6bee1c2 2023-02-13 12:15:44,592 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/9c79a36df690496ea7c4ff731fb5b039 2023-02-13 12:15:44,599 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/026bf871cef740b09aecf686e6bee1c2 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2 2023-02-13 12:15:44,605 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:15:44,607 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/9c79a36df690496ea7c4ff731fb5b039 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039 2023-02-13 12:15:44,617 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:15:44,618 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3bb9f1de2ea5395a2121754580ffb353 in 72ms, sequenceid=71, compaction requested=true 2023-02-13 12:15:44,618 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bb9f1de2ea5395a2121754580ffb353: 2023-02-13 12:15:44,618 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:44,618 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:44,618 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:44,618 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,619 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 12:15:44,619 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 12:15:44,626 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,626 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:44,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:44,626 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,626 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:44,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:44,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:44,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:44,628 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 12:15:44,628 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6597cdc8[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:44,628 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 12:15:44,628 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,634 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,634 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,634 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,634 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:44,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,634 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,634 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:44,634 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:44,634 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:44,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,635 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:44,635 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:44,635 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:44,635 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 12:15:44,635 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 12:15:44,635 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:44,642 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:44,642 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,643 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:44,651 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:44,651 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,651 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 12:15:44,651 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:44,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:44,651 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,651 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:44,651 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 12:15:44,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:15:44,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 12:15:44,654 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 12:15:44,654 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:44,654 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:44,654 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 12:15:44,654 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:44,665 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,665 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,665 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,665 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,665 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,665 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,665 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:44,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:44,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,667 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,668 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:44,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,668 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:44,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,670 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,670 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:44,670 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 12:15:44,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,681 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,682 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,682 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 12:15:44,682 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:44,682 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,682 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,682 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,682 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:44,682 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,682 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:44,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,682 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,682 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:44,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:44,682 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:44,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,683 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:44,683 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:44,683 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 12:15:44,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,683 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:44,683 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,684 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:44,684 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:44,684 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 12:15:44,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:44,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:44,684 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 12:15:44,783 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:44,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:45,055 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 12:15:45,058 DEBUG [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:45,060 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,061 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,061 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,061 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,062 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:45,062 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:45,062 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:45,310 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,312 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:45,312 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:45,312 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:45,315 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-13 12:15:45,316 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:45,316 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:45,316 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:45,316 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:45,317 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:45,321 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 12:15:45,323 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:45,324 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:45,324 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:45,325 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:45,325 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:45,326 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:45,326 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:45,327 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:45,328 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:45,328 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,329 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:45,329 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,329 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,329 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,330 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-13 12:15:45,332 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:45,333 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:45,340 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290575340 2023-02-13 12:15:45,342 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,342 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:45,347 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:45,347 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=3bb9f1de2ea5395a2121754580ffb353, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,45345,1676290533799, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799, openSeqNum=2 2023-02-13 12:15:45,347 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:45,348 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290605348 2023-02-13 12:15:45,348 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290665348 2023-02-13 12:15:45,348 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-13 12:15:45,348 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:45,351 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:45,351 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,351 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:45,351 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:45,598 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,598 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:45,598 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:45,598 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:45,598 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:45,599 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:45,599 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:45,599 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:45,599 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:45,600 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:45,601 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 12:15:45,602 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:45,603 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:45,603 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:45,603 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:45,604 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:45,604 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:45,605 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:45,605 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:45,605 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:45,606 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,606 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:45,606 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,606 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,606 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,607 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 12:15:45,607 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:45,608 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:45,608 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290575608 2023-02-13 12:15:45,610 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,610 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:45,614 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:45,614 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=3bb9f1de2ea5395a2121754580ffb353, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,45345,1676290533799, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799, openSeqNum=2 2023-02-13 12:15:45,614 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:45,614 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290605614 2023-02-13 12:15:45,615 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290665615 2023-02-13 12:15:45,615 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 12:15:45,615 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:45,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN}] 2023-02-13 12:15:45,618 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:45,619 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:45,619 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:45,865 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,865 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:45,865 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:45,866 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:45,866 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:45,866 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:45,866 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:45,866 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:45,866 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:45,867 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:45,869 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 12:15:45,869 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:45,870 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:45,870 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:45,870 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:45,871 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:45,872 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:45,872 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:45,873 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:45,873 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:45,874 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,874 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:45,874 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,874 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,874 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,874 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:15:45,875 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:45,875 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:45,876 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290575876 2023-02-13 12:15:45,877 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:45,877 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:45,880 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:45,880 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=3bb9f1de2ea5395a2121754580ffb353, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,45345,1676290533799, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799, openSeqNum=2 2023-02-13 12:15:45,880 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:45,880 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290605880 2023-02-13 12:15:45,881 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290665880 2023-02-13 12:15:45,881 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:45,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN}] 2023-02-13 12:15:45,881 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:45,883 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:45,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:45,884 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290545884"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290545884"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290545884"}]},"ts":"1676290545884"} 2023-02-13 12:15:45,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:15:45,886 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:45,886 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:45,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:45,886 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:46,121 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:15:46,131 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,131 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:46,131 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:46,131 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:46,131 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:46,132 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:46,132 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:46,132 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:46,132 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:46,134 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 2 msec 2023-02-13 12:15:46,136 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 12:15:46,137 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 12:15:46,137 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:46,137 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:46,138 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:46,138 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:46,139 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,139 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:46,140 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,140 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,140 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:46,142 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,142 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:46,142 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:46,142 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,142 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,143 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,143 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:46,143 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:46,143 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 12:15:46,144 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:46,144 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:46,145 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353 to restore RIT 2023-02-13 12:15:46,146 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290576146 2023-02-13 12:15:46,147 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:46,147 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:46,150 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:46,151 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSING, lastHost=jenkins-hbase16.apache.org,45345,1676290533799, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799, openSeqNum=2 2023-02-13 12:15:46,151 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:46,151 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290606151 2023-02-13 12:15:46,151 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290666151 2023-02-13 12:15:46,151 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:46,151 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:46,152 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290546151"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290546151"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290546151"}]},"ts":"1676290546151"} 2023-02-13 12:15:46,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:15:46,154 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:46,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 3bb9f1de2ea5395a2121754580ffb353, disabling compactions & flushes 2023-02-13 12:15:46,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:46,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:46,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. after waiting 0 ms 2023-02-13 12:15:46,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:46,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 3bb9f1de2ea5395a2121754580ffb353 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 12:15:46,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/44381de2ec884cc7ae771f5467029124 2023-02-13 12:15:46,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/871e4fcaa5694c5abb22eec550836f27 2023-02-13 12:15:46,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf1/44381de2ec884cc7ae771f5467029124 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124 2023-02-13 12:15:46,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:15:46,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/.tmp/cf2/871e4fcaa5694c5abb22eec550836f27 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27 2023-02-13 12:15:46,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:15:46,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3bb9f1de2ea5395a2121754580ffb353 in 58ms, sequenceid=82, compaction requested=true 2023-02-13 12:15:46,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 12:15:46,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:15:46,391 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. 2023-02-13 12:15:46,391 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 3bb9f1de2ea5395a2121754580ffb353: Waiting for close lock at 1676290546310Running coprocessor pre-close hooks at 1676290546310Disabling compacts and flushes for region at 1676290546310Disabling writes for close at 1676290546310Obtaining lock to block concurrent updates at 1676290546310Preparing flush snapshotting stores in 3bb9f1de2ea5395a2121754580ffb353 at 1676290546310Finished memstore snapshotting testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676290546311 (+1 ms)Flushing stores of testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. at 1676290546311Flushing 3bb9f1de2ea5395a2121754580ffb353/cf1: creating writer at 1676290546311Flushing 3bb9f1de2ea5395a2121754580ffb353/cf1: appending metadata at 1676290546313 (+2 ms)Flushing 3bb9f1de2ea5395a2121754580ffb353/cf1: closing flushed file at 1676290546314 (+1 ms)Flushing 3bb9f1de2ea5395a2121754580ffb353/cf2: creating writer at 1676290546331 (+17 ms)Flushing 3bb9f1de2ea5395a2121754580ffb353/cf2: appending metadata at 1676290546333 (+2 ms)Flushing 3bb9f1de2ea5395a2121754580ffb353/cf2: closing flushed file at 1676290546333Flushing 3bb9f1de2ea5395a2121754580ffb353/cf1: reopening flushed file at 1676290546356 (+23 ms)Flushing 3bb9f1de2ea5395a2121754580ffb353/cf2: reopening flushed file at 1676290546362 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3bb9f1de2ea5395a2121754580ffb353 in 58ms, sequenceid=82, compaction requested=true at 1676290546368 (+6 ms)Writing region close event to WAL at 1676290546386 (+18 ms)Running coprocessor post-close hooks at 1676290546390 (+4 ms)Closed at 1676290546391 (+1 ms) 2023-02-13 12:15:46,394 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,395 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSED 2023-02-13 12:15:46,395 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290546395"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290546395"}]},"ts":"1676290546395"} 2023-02-13 12:15:46,397 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:46,397 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:46,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:46,397 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:46,401 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,401 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:46,401 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:46,402 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:46,402 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:46,402 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:46,402 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:46,402 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:46,402 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:46,403 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:46,404 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 12:15:46,404 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 12:15:46,404 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 12:15:46,405 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:46,405 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:46,405 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:46,405 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:46,406 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,406 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:46,406 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,406 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,407 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:46,408 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,408 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:46,408 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:46,408 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:46,408 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,408 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,408 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,408 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:46,409 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:46,409 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:46,409 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 12:15:46,409 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:46,410 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:46,410 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353 to restore RIT 2023-02-13 12:15:46,410 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290576410 2023-02-13 12:15:46,411 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:46,411 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:46,414 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:46,414 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,45345,1676290533799, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799, openSeqNum=2 2023-02-13 12:15:46,414 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:46,414 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290606414 2023-02-13 12:15:46,414 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290666414 2023-02-13 12:15:46,414 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:46,415 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSED, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:46,415 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290546414"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290546414"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290546414"}]},"ts":"1676290546414"} 2023-02-13 12:15:46,416 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:46,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-13 12:15:46,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 in 262 msec 2023-02-13 12:15:46,439 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:46,439 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:46,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:46,439 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:46,665 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,665 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:46,665 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:46,665 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:46,666 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:46,666 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:46,666 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:46,666 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:46,666 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:46,667 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:46,669 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 12:15:46,669 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 12:15:46,669 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:46,669 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:46,670 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:46,670 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:46,670 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:46,670 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,671 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:46,671 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,671 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,671 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:46,672 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,672 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:46,672 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:46,672 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:46,672 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,672 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,672 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,672 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:46,673 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:46,673 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 12:15:46,673 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:46,674 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:46,674 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353 to restore RIT 2023-02-13 12:15:46,674 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290576674 2023-02-13 12:15:46,674 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:46,675 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:46,678 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:46,678 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,45345,1676290533799, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799, openSeqNum=2 2023-02-13 12:15:46,678 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:46,678 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290606678 2023-02-13 12:15:46,678 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290666678 2023-02-13 12:15:46,679 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 12:15:46,679 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:46,681 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:46,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-13 12:15:46,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN in 800 msec 2023-02-13 12:15:46,695 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:46,695 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:46,695 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:46,929 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,929 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:46,929 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:46,929 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:46,930 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:46,930 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:46,931 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:46,931 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:46,931 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:46,933 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:46,936 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 12:15:46,937 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:46,937 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:46,938 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:46,939 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:46,939 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:46,940 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:46,940 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,940 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:46,940 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,940 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:46,941 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:46,941 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,941 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:46,941 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:46,941 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:46,942 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:46,942 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:46,942 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 12:15:46,942 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:46,943 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:46,943 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290576943 2023-02-13 12:15:46,944 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:46,945 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,947 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:46,947 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,45345,1676290533799, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799, openSeqNum=2 2023-02-13 12:15:46,947 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:46,947 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290606947 2023-02-13 12:15:46,947 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290666947 2023-02-13 12:15:46,947 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:46,947 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,948 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,948 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:46,953 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:46,964 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=3bb9f1de2ea5395a2121754580ffb353, threads=14 2023-02-13 12:15:46,965 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,965 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,965 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,965 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,966 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,966 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,966 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,966 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,966 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,968 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,970 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,971 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,971 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:46,971 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,087 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,092 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,096 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,112 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,119 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,120 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,123 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,128 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,128 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,129 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,129 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,129 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,129 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,129 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,129 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 3bb9f1de2ea5395a2121754580ffb353 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 12:15:47,172 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:47,172 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:47,172 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:47,198 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,198 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:47,198 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:47,198 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:47,198 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:47,198 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:47,199 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:47,199 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:47,199 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:47,200 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:47,201 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 12:15:47,201 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:47,201 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:47,202 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:47,203 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:47,203 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:47,203 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:47,204 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:47,204 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:47,205 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:47,205 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:47,206 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:47,207 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,207 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:47,207 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:47,207 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:47,207 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,208 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,208 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,208 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:47,208 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:47,208 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 12:15:47,209 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:47,209 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:47,210 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290577210 2023-02-13 12:15:47,211 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,211 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:47,214 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:47,215 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,45345,1676290533799, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799, openSeqNum=2 2023-02-13 12:15:47,215 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:47,215 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290607215 2023-02-13 12:15:47,215 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290667215 2023-02-13 12:15:47,215 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:47,224 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=3bb9f1de2ea5395a2121754580ffb353, threads=14 2023-02-13 12:15:47,225 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,225 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,225 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,226 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,228 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,232 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,232 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,232 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,236 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,236 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,236 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,238 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,239 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,239 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,448 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,484 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,521 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,521 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,536 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,548 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,548 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,549 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,552 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,552 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,556 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,560 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,560 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,566 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2 for region: 3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:47,567 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 3bb9f1de2ea5395a2121754580ffb353 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 12:15:47,612 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:47,619 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:15:47,621 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:15:47,621 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:47,621 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:47,621 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:47,716 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,716 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:47,716 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:47,716 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:47,717 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:47,717 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:47,717 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:47,717 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:47,717 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:47,718 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:47,720 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 12:15:47,720 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:47,720 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:47,720 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:47,721 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:47,721 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:47,721 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:47,722 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:47,722 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:47,722 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:47,723 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:47,723 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:47,724 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,724 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:47,724 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:47,724 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:47,724 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,725 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,725 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,725 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:47,725 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:47,725 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 12:15:47,726 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:47,726 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:47,727 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290577727 2023-02-13 12:15:47,728 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,728 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:47,731 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:47,731 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,45345,1676290533799, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799, openSeqNum=2 2023-02-13 12:15:47,731 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:47,731 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290607731 2023-02-13 12:15:47,732 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290667732 2023-02-13 12:15:47,732 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:47,735 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:47,737 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:47,737 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:47,737 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:47,982 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:15:47,982 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:47,982 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:47,982 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:47,982 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:47,983 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:47,983 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:47,983 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:47,983 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:47,983 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:47,984 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 12:15:47,984 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:47,984 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:47,985 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:47,985 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:47,985 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:47,985 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:47,986 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:47,986 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:47,986 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:47,986 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:47,986 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:47,987 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,987 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN 2023-02-13 12:15:47,987 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:47,987 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:47,987 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,987 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,987 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,988 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3bb9f1de2ea5395a2121754580ffb353, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:47,988 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 3bb9f1de2ea5395a2121754580ffb353, server=jenkins-hbase16.apache.org,45345,1676290533799 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:47,988 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 12:15:47,988 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:47,989 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:47,989 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290577989 2023-02-13 12:15:47,991 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:47,991 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:47,993 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:47,993 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=3bb9f1de2ea5395a2121754580ffb353, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,45345,1676290533799, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799, openSeqNum=2 2023-02-13 12:15:47,993 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:47,993 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290607993 2023-02-13 12:15:47,993 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290667993 2023-02-13 12:15:47,993 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 12:15:47,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290547997"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676290547997"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676290547997"}]},"ts":"1676290547997"} 2023-02-13 12:15:47,998 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290547997"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290547997"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290547997"}]},"ts":"1676290547997"} 2023-02-13 12:15:47,998 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290547997"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290547997"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290547997"}]},"ts":"1676290547997"} 2023-02-13 12:15:48,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=35372d8b400d126b69effe8c26d81515, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a9394292bc62d1bc8ddb07cb3a18edde, ASSIGN}] 2023-02-13 12:15:48,005 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a9394292bc62d1bc8ddb07cb3a18edde, ASSIGN 2023-02-13 12:15:48,006 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a9394292bc62d1bc8ddb07cb3a18edde, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,45345,1676290533799; forceNewPlan=false, retain=false 2023-02-13 12:15:48,007 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=35372d8b400d126b69effe8c26d81515, ASSIGN 2023-02-13 12:15:48,008 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=35372d8b400d126b69effe8c26d81515, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,45345,1676290533799; forceNewPlan=false, retain=false 2023-02-13 12:15:48,156 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 12:15:48,158 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=a9394292bc62d1bc8ddb07cb3a18edde, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:48,158 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290548158"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290548158"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290548158"}]},"ts":"1676290548158"} 2023-02-13 12:15:48,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure a9394292bc62d1bc8ddb07cb3a18edde, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:15:48,163 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=35372d8b400d126b69effe8c26d81515, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:48,163 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290548163"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290548163"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290548163"}]},"ts":"1676290548163"} 2023-02-13 12:15:48,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 35372d8b400d126b69effe8c26d81515, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:15:48,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:48,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a9394292bc62d1bc8ddb07cb3a18edde, NAME => 'testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 12:15:48,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:48,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:48,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:48,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:48,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:48,321 INFO [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:48,323 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1 2023-02-13 12:15:48,323 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1 2023-02-13 12:15:48,323 INFO [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9394292bc62d1bc8ddb07cb3a18edde columnFamilyName cf1 2023-02-13 12:15:48,343 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2-top 2023-02-13 12:15:48,348 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec-top 2023-02-13 12:15:48,354 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124-top 2023-02-13 12:15:48,360 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813-top 2023-02-13 12:15:48,367 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa-top 2023-02-13 12:15:48,372 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111-top 2023-02-13 12:15:48,377 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2-top 2023-02-13 12:15:48,377 INFO [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(310): Store=a9394292bc62d1bc8ddb07cb3a18edde/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:48,377 INFO [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:48,379 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2 2023-02-13 12:15:48,379 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2 2023-02-13 12:15:48,379 INFO [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9394292bc62d1bc8ddb07cb3a18edde columnFamilyName cf2 2023-02-13 12:15:48,400 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557-top 2023-02-13 12:15:48,407 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0-top 2023-02-13 12:15:48,412 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108-top 2023-02-13 12:15:48,417 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27-top 2023-02-13 12:15:48,424 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b-top 2023-02-13 12:15:48,430 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039-top 2023-02-13 12:15:48,436 DEBUG [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24-top 2023-02-13 12:15:48,437 INFO [StoreOpener-a9394292bc62d1bc8ddb07cb3a18edde-1] regionserver.HStore(310): Store=a9394292bc62d1bc8ddb07cb3a18edde/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:48,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:48,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:48,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:15:48,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:48,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened a9394292bc62d1bc8ddb07cb3a18edde; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61695460, jitterRate=-0.08066600561141968}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:15:48,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for a9394292bc62d1bc8ddb07cb3a18edde: Running coprocessor pre-open hook at 1676290548319Writing region info on filesystem at 1676290548319Initializing all the Stores at 1676290548321 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290548321Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290548321Cleaning up temporary data from old regions at 1676290548439 (+118 ms)Cleaning up detritus from prior splits at 1676290548440 (+1 ms)Running coprocessor post-open hooks at 1676290548443 (+3 ms)Region opened successfully at 1676290548443 2023-02-13 12:15:48,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde., pid=33, masterSystemTime=1676290548312 2023-02-13 12:15:48,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:48,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:15:48,445 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:15:48,445 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:15:48,445 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:48,445 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:48,445 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HStore(1912): a9394292bc62d1bc8ddb07cb3a18edde/cf2 is initiating minor compaction (all files) 2023-02-13 12:15:48,445 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HStore(1912): a9394292bc62d1bc8ddb07cb3a18edde/cf1 is initiating minor compaction (all files) 2023-02-13 12:15:48,445 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a9394292bc62d1bc8ddb07cb3a18edde/cf2 in testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:48,445 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a9394292bc62d1bc8ddb07cb3a18edde/cf1 in testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:48,446 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27-top] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/.tmp, totalSize=34.9 K 2023-02-13 12:15:48,446 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124-top] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/.tmp, totalSize=34.9 K 2023-02-13 12:15:48,446 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676290542916 2023-02-13 12:15:48,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:48,446 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676290542916 2023-02-13 12:15:48,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:48,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:48,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 35372d8b400d126b69effe8c26d81515, NAME => 'testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 12:15:48,447 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676290543181 2023-02-13 12:15:48,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:48,447 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676290543181 2023-02-13 12:15:48,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:48,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:48,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:48,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:48,448 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676290543549 2023-02-13 12:15:48,448 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=a9394292bc62d1bc8ddb07cb3a18edde, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:48,449 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290548448"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290548448"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290548448"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290548448"}]},"ts":"1676290548448"} 2023-02-13 12:15:48,449 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676290543549 2023-02-13 12:15:48,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-13 12:15:48,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure a9394292bc62d1bc8ddb07cb3a18edde, server=jenkins-hbase16.apache.org,45345,1676290533799 in 290 msec 2023-02-13 12:15:48,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a9394292bc62d1bc8ddb07cb3a18edde, ASSIGN in 450 msec 2023-02-13 12:15:48,456 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676290543861 2023-02-13 12:15:48,456 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676290543861 2023-02-13 12:15:48,457 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676290544155 2023-02-13 12:15:48,457 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676290544155 2023-02-13 12:15:48,457 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676290544513 2023-02-13 12:15:48,458 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676290544513 2023-02-13 12:15:48,458 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676290544786 2023-02-13 12:15:48,458 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676290544786 2023-02-13 12:15:48,461 INFO [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:48,462 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1 2023-02-13 12:15:48,462 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1 2023-02-13 12:15:48,462 INFO [StoreOpener-35372d8b400d126b69effe8c26d81515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35372d8b400d126b69effe8c26d81515 columnFamilyName cf1 2023-02-13 12:15:48,486 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2-bottom 2023-02-13 12:15:48,489 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] throttle.PressureAwareThroughputController(145): a9394292bc62d1bc8ddb07cb3a18edde#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:15:48,490 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec-bottom 2023-02-13 12:15:48,493 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] throttle.PressureAwareThroughputController(145): a9394292bc62d1bc8ddb07cb3a18edde#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:15:48,501 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124-bottom 2023-02-13 12:15:48,514 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813-bottom 2023-02-13 12:15:48,520 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa-bottom 2023-02-13 12:15:48,523 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/.tmp/cf2/032f5067cc81412d9305c545d13b9e9b as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/032f5067cc81412d9305c545d13b9e9b 2023-02-13 12:15:48,523 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/.tmp/cf1/3048b8b293d9443eaaed957dbe2206a3 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/3048b8b293d9443eaaed957dbe2206a3 2023-02-13 12:15:48,528 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111-bottom 2023-02-13 12:15:48,535 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2-bottom 2023-02-13 12:15:48,536 INFO [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(310): Store=35372d8b400d126b69effe8c26d81515/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:48,536 INFO [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:48,536 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a9394292bc62d1bc8ddb07cb3a18edde/cf2 of a9394292bc62d1bc8ddb07cb3a18edde into 032f5067cc81412d9305c545d13b9e9b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:15:48,537 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a9394292bc62d1bc8ddb07cb3a18edde: 2023-02-13 12:15:48,537 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde., storeName=a9394292bc62d1bc8ddb07cb3a18edde/cf2, priority=9, startTime=1676290548444; duration=0sec 2023-02-13 12:15:48,537 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:48,537 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2 2023-02-13 12:15:48,537 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2 2023-02-13 12:15:48,538 INFO [StoreOpener-35372d8b400d126b69effe8c26d81515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35372d8b400d126b69effe8c26d81515 columnFamilyName cf2 2023-02-13 12:15:48,538 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a9394292bc62d1bc8ddb07cb3a18edde/cf1 of a9394292bc62d1bc8ddb07cb3a18edde into 3048b8b293d9443eaaed957dbe2206a3(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:15:48,538 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a9394292bc62d1bc8ddb07cb3a18edde: 2023-02-13 12:15:48,538 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde., storeName=a9394292bc62d1bc8ddb07cb3a18edde/cf1, priority=9, startTime=1676290548444; duration=0sec 2023-02-13 12:15:48,538 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:48,559 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557-bottom 2023-02-13 12:15:48,564 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0-bottom 2023-02-13 12:15:48,568 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108-bottom 2023-02-13 12:15:48,573 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27-bottom 2023-02-13 12:15:48,576 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b-bottom 2023-02-13 12:15:48,587 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039-bottom 2023-02-13 12:15:48,592 DEBUG [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24-bottom 2023-02-13 12:15:48,592 INFO [StoreOpener-35372d8b400d126b69effe8c26d81515-1] regionserver.HStore(310): Store=35372d8b400d126b69effe8c26d81515/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:48,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:48,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:48,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:15:48,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:48,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 35372d8b400d126b69effe8c26d81515; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69326538, jitterRate=0.03304591774940491}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:15:48,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 35372d8b400d126b69effe8c26d81515: Running coprocessor pre-open hook at 1676290548448Writing region info on filesystem at 1676290548448Initializing all the Stores at 1676290548456 (+8 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290548456Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290548461 (+5 ms)Cleaning up temporary data from old regions at 1676290548595 (+134 ms)Cleaning up detritus from prior splits at 1676290548595Running coprocessor post-open hooks at 1676290548598 (+3 ms)Region opened successfully at 1676290548598 2023-02-13 12:15:48,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515., pid=34, masterSystemTime=1676290548312 2023-02-13 12:15:48,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:48,599 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:15:48,599 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:15:48,600 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:48,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:48,600 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:48,600 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HStore(1912): 35372d8b400d126b69effe8c26d81515/cf1 is initiating minor compaction (all files) 2023-02-13 12:15:48,600 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HStore(1912): 35372d8b400d126b69effe8c26d81515/cf2 is initiating minor compaction (all files) 2023-02-13 12:15:48,600 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 35372d8b400d126b69effe8c26d81515/cf1 in testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:48,600 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 35372d8b400d126b69effe8c26d81515/cf2 in testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:48,600 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27-bottom] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/.tmp, totalSize=34.9 K 2023-02-13 12:15:48,600 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124-bottom] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/.tmp, totalSize=34.9 K 2023-02-13 12:15:48,601 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290542916 2023-02-13 12:15:48,601 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290542916 2023-02-13 12:15:48,601 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290543181 2023-02-13 12:15:48,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:48,601 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290543181 2023-02-13 12:15:48,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:48,602 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290543549 2023-02-13 12:15:48,602 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=35372d8b400d126b69effe8c26d81515, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:48,602 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290543549 2023-02-13 12:15:48,602 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290548602"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290548602"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290548602"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290548602"}]},"ts":"1676290548602"} 2023-02-13 12:15:48,603 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290543861 2023-02-13 12:15:48,603 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290543861 2023-02-13 12:15:48,603 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290544155 2023-02-13 12:15:48,603 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290544155 2023-02-13 12:15:48,604 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290544513 2023-02-13 12:15:48,604 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290544513 2023-02-13 12:15:48,604 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] compactions.Compactor(207): Compacting 871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290544786 2023-02-13 12:15:48,604 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] compactions.Compactor(207): Compacting 44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290544786 2023-02-13 12:15:48,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-13 12:15:48,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 35372d8b400d126b69effe8c26d81515, server=jenkins-hbase16.apache.org,45345,1676290533799 in 440 msec 2023-02-13 12:15:48,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-13 12:15:48,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=35372d8b400d126b69effe8c26d81515, ASSIGN in 607 msec 2023-02-13 12:15:48,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde in 3.5550 sec 2023-02-13 12:15:48,622 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] throttle.PressureAwareThroughputController(145): 35372d8b400d126b69effe8c26d81515#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:15:48,623 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] throttle.PressureAwareThroughputController(145): 35372d8b400d126b69effe8c26d81515#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:15:48,649 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/.tmp/cf1/6fb44a7a86244fd2b209bcab3ce874a8 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/6fb44a7a86244fd2b209bcab3ce874a8 2023-02-13 12:15:48,653 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/.tmp/cf2/50cc2f2f88e24703a483f359e5f5508c as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/50cc2f2f88e24703a483f359e5f5508c 2023-02-13 12:15:48,658 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 35372d8b400d126b69effe8c26d81515/cf1 of 35372d8b400d126b69effe8c26d81515 into 6fb44a7a86244fd2b209bcab3ce874a8(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:15:48,658 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 35372d8b400d126b69effe8c26d81515: 2023-02-13 12:15:48,658 INFO [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515., storeName=35372d8b400d126b69effe8c26d81515/cf1, priority=9, startTime=1676290548599; duration=0sec 2023-02-13 12:15:48,658 DEBUG [RS:2;jenkins-hbase16:45345-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:48,660 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 35372d8b400d126b69effe8c26d81515/cf2 of 35372d8b400d126b69effe8c26d81515 into 50cc2f2f88e24703a483f359e5f5508c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:15:48,660 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 35372d8b400d126b69effe8c26d81515: 2023-02-13 12:15:48,660 INFO [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515., storeName=35372d8b400d126b69effe8c26d81515/cf2, priority=9, startTime=1676290548599; duration=0sec 2023-02-13 12:15:48,660 DEBUG [RS:2;jenkins-hbase16:45345-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:48,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45345] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 188.40.62.62:50692 deadline: 1676290608744, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353. is not online on jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:48,858 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 12:15:48,869 INFO [Listener at localhost.localdomain/46087] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 12:15:48,871 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 12:15:48,899 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:15:48,901 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-13 12:15:48,902 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitWithoutPONR 2023-02-13 12:15:48,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:48,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 12:15:48,907 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290548907"}]},"ts":"1676290548907"} 2023-02-13 12:15:48,909 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-13 12:15:48,917 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-13 12:15:48,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=35372d8b400d126b69effe8c26d81515, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a9394292bc62d1bc8ddb07cb3a18edde, UNASSIGN}] 2023-02-13 12:15:48,919 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a9394292bc62d1bc8ddb07cb3a18edde, UNASSIGN 2023-02-13 12:15:48,920 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=a9394292bc62d1bc8ddb07cb3a18edde, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:48,920 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290548920"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290548920"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290548920"}]},"ts":"1676290548920"} 2023-02-13 12:15:48,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure a9394292bc62d1bc8ddb07cb3a18edde, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:15:48,926 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=35372d8b400d126b69effe8c26d81515, UNASSIGN 2023-02-13 12:15:48,927 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=35372d8b400d126b69effe8c26d81515, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:48,927 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290548927"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290548927"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290548927"}]},"ts":"1676290548927"} 2023-02-13 12:15:48,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 35372d8b400d126b69effe8c26d81515, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:15:49,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 12:15:49,075 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:49,075 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 35372d8b400d126b69effe8c26d81515, disabling compactions & flushes 2023-02-13 12:15:49,075 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:49,075 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:49,075 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. after waiting 0 ms 2023-02-13 12:15:49,075 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:49,085 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124-bottom] to archive 2023-02-13 12:15:49,086 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:15:49,088 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,089 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,091 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,092 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,094 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,095 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,096 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,097 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27-bottom] to archive 2023-02-13 12:15:49,098 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:15:49,099 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,101 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,102 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,103 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,104 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,105 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,106 DEBUG [StoreCloser-testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,110 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 12:15:49,110 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:15:49,110 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515. 2023-02-13 12:15:49,110 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 35372d8b400d126b69effe8c26d81515: Waiting for close lock at 1676290549075Running coprocessor pre-close hooks at 1676290549075Disabling compacts and flushes for region at 1676290549075Disabling writes for close at 1676290549075Writing region close event to WAL at 1676290549107 (+32 ms)Running coprocessor post-close hooks at 1676290549110 (+3 ms)Closed at 1676290549110 2023-02-13 12:15:49,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:49,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:49,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing a9394292bc62d1bc8ddb07cb3a18edde, disabling compactions & flushes 2023-02-13 12:15:49,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:49,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:49,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. after waiting 0 ms 2023-02-13 12:15:49,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:49,113 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=35372d8b400d126b69effe8c26d81515, regionState=CLOSED 2023-02-13 12:15:49,113 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290549113"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290549113"}]},"ts":"1676290549113"} 2023-02-13 12:15:49,114 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124-top] to archive 2023-02-13 12:15:49,114 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:15:49,116 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-13 12:15:49,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 35372d8b400d126b69effe8c26d81515, server=jenkins-hbase16.apache.org,45345,1676290533799 in 186 msec 2023-02-13 12:15:49,118 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=35372d8b400d126b69effe8c26d81515, UNASSIGN in 199 msec 2023-02-13 12:15:49,119 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,120 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,122 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,123 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,124 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,126 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27-top] to archive 2023-02-13 12:15:49,127 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:15:49,129 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,130 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,131 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,133 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,134 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,135 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,136 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 12:15:49,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:15:49,142 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde. 2023-02-13 12:15:49,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for a9394292bc62d1bc8ddb07cb3a18edde: Waiting for close lock at 1676290549112Running coprocessor pre-close hooks at 1676290549112Disabling compacts and flushes for region at 1676290549112Disabling writes for close at 1676290549113 (+1 ms)Writing region close event to WAL at 1676290549139 (+26 ms)Running coprocessor post-close hooks at 1676290549142 (+3 ms)Closed at 1676290549142 2023-02-13 12:15:49,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:49,145 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=a9394292bc62d1bc8ddb07cb3a18edde, regionState=CLOSED 2023-02-13 12:15:49,145 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290549145"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290549145"}]},"ts":"1676290549145"} 2023-02-13 12:15:49,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-13 12:15:49,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure a9394292bc62d1bc8ddb07cb3a18edde, server=jenkins-hbase16.apache.org,45345,1676290533799 in 224 msec 2023-02-13 12:15:49,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-13 12:15:49,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a9394292bc62d1bc8ddb07cb3a18edde, UNASSIGN in 230 msec 2023-02-13 12:15:49,151 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290549151"}]},"ts":"1676290549151"} 2023-02-13 12:15:49,152 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-13 12:15:49,159 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-13 12:15:49,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 258 msec 2023-02-13 12:15:49,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 12:15:49,210 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-13 12:15:49,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitWithoutPONR 2023-02-13 12:15:49,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:49,215 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:49,216 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:49,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 12:15:49,225 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,225 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:49,225 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:49,227 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/recovered.edits] 2023-02-13 12:15:49,227 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/recovered.edits] 2023-02-13 12:15:49,228 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/recovered.edits] 2023-02-13 12:15:49,231 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/.regioninfo to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/.regioninfo 2023-02-13 12:15:49,231 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/.regioninfo to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/.regioninfo 2023-02-13 12:15:49,231 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/026bf871cef740b09aecf686e6bee1c2 2023-02-13 12:15:49,233 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/146d5b7be2d94cafb42bbd42cb0f65ec 2023-02-13 12:15:49,235 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,235 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/44381de2ec884cc7ae771f5467029124 2023-02-13 12:15:49,235 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/026bf871cef740b09aecf686e6bee1c2.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,237 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/47ab6e8111334a9690a1befe29f29813 2023-02-13 12:15:49,237 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,237 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/146d5b7be2d94cafb42bbd42cb0f65ec.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,238 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/5d8c1c02e42149039cb6615b911d54fa 2023-02-13 12:15:49,238 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,238 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/44381de2ec884cc7ae771f5467029124.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,239 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/a7306eb0442640abaf8bfc64a0e46111 2023-02-13 12:15:49,239 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,239 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/47ab6e8111334a9690a1befe29f29813.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,241 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,241 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf1/c29367d79c0a4e1d98a2733e505eb6d2 2023-02-13 12:15:49,241 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/5d8c1c02e42149039cb6615b911d54fa.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,242 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,242 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/a7306eb0442640abaf8bfc64a0e46111.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,243 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,244 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/0759008837e747bb91986c986dd15557 2023-02-13 12:15:49,244 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf1/c29367d79c0a4e1d98a2733e505eb6d2.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,246 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/11e2cc96f05e422aa075e4fae58d4ac0 2023-02-13 12:15:49,248 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,248 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/33c1bc2d753b44499adbadcc8bd66108 2023-02-13 12:15:49,250 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,251 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,252 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,252 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/0759008837e747bb91986c986dd15557.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,253 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/871e4fcaa5694c5abb22eec550836f27 2023-02-13 12:15:49,253 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,254 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/11e2cc96f05e422aa075e4fae58d4ac0.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,254 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/8ba6dd0a1459467fb94a960d2708ca2b 2023-02-13 12:15:49,255 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,255 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/33c1bc2d753b44499adbadcc8bd66108.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,257 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/9c79a36df690496ea7c4ff731fb5b039 2023-02-13 12:15:49,257 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/a9394292bc62d1bc8ddb07cb3a18edde/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,258 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/871e4fcaa5694c5abb22eec550836f27.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,261 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/cf2/a70f882bb2ec496ead2f96d700399c24 2023-02-13 12:15:49,261 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/8ba6dd0a1459467fb94a960d2708ca2b.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,261 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/3048b8b293d9443eaaed957dbe2206a3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf1/3048b8b293d9443eaaed957dbe2206a3 2023-02-13 12:15:49,265 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/9c79a36df690496ea7c4ff731fb5b039.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,266 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/recovered.edits/85.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353/recovered.edits/85.seqid 2023-02-13 12:15:49,266 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/35372d8b400d126b69effe8c26d81515/cf2/a70f882bb2ec496ead2f96d700399c24.3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,267 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/032f5067cc81412d9305c545d13b9e9b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/cf2/032f5067cc81412d9305c545d13b9e9b 2023-02-13 12:15:49,267 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/3bb9f1de2ea5395a2121754580ffb353 2023-02-13 12:15:49,269 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/6fb44a7a86244fd2b209bcab3ce874a8 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf1/6fb44a7a86244fd2b209bcab3ce874a8 2023-02-13 12:15:49,270 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/recovered.edits/90.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde/recovered.edits/90.seqid 2023-02-13 12:15:49,271 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:49,273 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/50cc2f2f88e24703a483f359e5f5508c to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/cf2/50cc2f2f88e24703a483f359e5f5508c 2023-02-13 12:15:49,278 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/recovered.edits/90.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515/recovered.edits/90.seqid 2023-02-13 12:15:49,280 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitWithoutPONR/35372d8b400d126b69effe8c26d81515 2023-02-13 12:15:49,280 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 12:15:49,283 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:49,285 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-13 12:15:49,288 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-13 12:15:49,289 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:49,289 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-13 12:15:49,290 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290549289"}]},"ts":"9223372036854775807"} 2023-02-13 12:15:49,290 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290549289"}]},"ts":"9223372036854775807"} 2023-02-13 12:15:49,290 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290549289"}]},"ts":"9223372036854775807"} 2023-02-13 12:15:49,292 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 12:15:49,292 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3bb9f1de2ea5395a2121754580ffb353, NAME => 'testSplitWithoutPONR,,1676290542157.3bb9f1de2ea5395a2121754580ffb353.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 35372d8b400d126b69effe8c26d81515, NAME => 'testSplitWithoutPONR,,1676290545058.35372d8b400d126b69effe8c26d81515.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => a9394292bc62d1bc8ddb07cb3a18edde, NAME => 'testSplitWithoutPONR,41,1676290545058.a9394292bc62d1bc8ddb07cb3a18edde.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 12:15:49,293 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-13 12:15:49,293 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676290549293"}]},"ts":"9223372036854775807"} 2023-02-13 12:15:49,294 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-13 12:15:49,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 12:15:49,357 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:49,358 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 146 msec 2023-02-13 12:15:49,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 12:15:49,520 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-13 12:15:49,535 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=404 (was 394) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:34262 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:34858 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:34874 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:49094 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3fbde3a3-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:42790 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:34056 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:49306 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:48322 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:34860 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:49106 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:48336 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:34216 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:49330 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:34080 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:34042 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase16:45345-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1868338742_15 at /127.0.0.1:48482 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=876 (was 835) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=260 (was 247) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=5640 (was 6251) 2023-02-13 12:15:49,551 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=404, OpenFileDescriptor=876, MaxFileDescriptor=60000, SystemLoadAverage=260, ProcessCount=167, AvailableMemoryMB=5639 2023-02-13 12:15:49,551 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:15:49,636 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 12:15:49,637 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 12:15:49,638 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:49,640 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:15:49,641 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:15:49,642 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:49,643 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e empty. 2023-02-13 12:15:49,643 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:49,644 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 12:15:49,658 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 12:15:49,659 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => c234839e44de2733ae92a93b34e9e56e, NAME => 'testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:15:49,671 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:49,671 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing c234839e44de2733ae92a93b34e9e56e, disabling compactions & flushes 2023-02-13 12:15:49,671 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:49,671 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:49,671 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. after waiting 0 ms 2023-02-13 12:15:49,671 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:49,671 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:49,671 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for c234839e44de2733ae92a93b34e9e56e: Waiting for close lock at 1676290549671Disabling compacts and flushes for region at 1676290549671Disabling writes for close at 1676290549671Writing region close event to WAL at 1676290549671Closed at 1676290549671 2023-02-13 12:15:49,673 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:15:49,674 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290549674"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290549674"}]},"ts":"1676290549674"} 2023-02-13 12:15:49,676 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:15:49,677 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:15:49,677 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290549677"}]},"ts":"1676290549677"} 2023-02-13 12:15:49,678 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 12:15:49,741 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:15:49,743 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:15:49,743 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:15:49,743 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:15:49,743 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:15:49,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN}] 2023-02-13 12:15:49,754 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:49,755 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,33961,1676290533759; forceNewPlan=false, retain=false 2023-02-13 12:15:49,906 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:15:49,907 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:49,907 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290549907"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290549907"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290549907"}]},"ts":"1676290549907"} 2023-02-13 12:15:49,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:15:50,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:50,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c234839e44de2733ae92a93b34e9e56e, NAME => 'testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:15:50,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:50,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:50,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:50,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:50,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:50,071 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:50,072 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1 2023-02-13 12:15:50,072 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1 2023-02-13 12:15:50,073 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c234839e44de2733ae92a93b34e9e56e columnFamilyName cf1 2023-02-13 12:15:50,073 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(310): Store=c234839e44de2733ae92a93b34e9e56e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:50,074 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:50,075 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2 2023-02-13 12:15:50,075 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2 2023-02-13 12:15:50,076 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c234839e44de2733ae92a93b34e9e56e columnFamilyName cf2 2023-02-13 12:15:50,077 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(310): Store=c234839e44de2733ae92a93b34e9e56e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:50,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:50,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:50,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:15:50,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:50,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:15:50,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened c234839e44de2733ae92a93b34e9e56e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65670716, jitterRate=-0.02143007516860962}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:15:50,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for c234839e44de2733ae92a93b34e9e56e: Running coprocessor pre-open hook at 1676290550070Writing region info on filesystem at 1676290550070Initializing all the Stores at 1676290550070Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290550070Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290550070Cleaning up temporary data from old regions at 1676290550080 (+10 ms)Cleaning up detritus from prior splits at 1676290550080Running coprocessor post-open hooks at 1676290550106 (+26 ms)Region opened successfully at 1676290550106 2023-02-13 12:15:50,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e., pid=43, masterSystemTime=1676290550061 2023-02-13 12:15:50,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:50,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:50,110 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,110 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290550109"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290550109"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290550109"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290550109"}]},"ts":"1676290550109"} 2023-02-13 12:15:50,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-13 12:15:50,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 in 209 msec 2023-02-13 12:15:50,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-13 12:15:50,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN in 382 msec 2023-02-13 12:15:50,136 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:15:50,136 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290550136"}]},"ts":"1676290550136"} 2023-02-13 12:15:50,138 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 12:15:50,149 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:15:50,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 512 msec 2023-02-13 12:15:50,398 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:50,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 12:15:50,400 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 12:15:50,400 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:50,400 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 12:15:50,400 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:15:50,401 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,401 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:50,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:50,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:50,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:50,409 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,409 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:50,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,410 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,410 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:50,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:50,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:50,411 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:50,411 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:50,411 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:50,412 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:50,413 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:50,413 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. started... 2023-02-13 12:15:50,414 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c234839e44de2733ae92a93b34e9e56e 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 12:15:50,422 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,428 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/bf44b77c2db742148b73ef8d840bb392 2023-02-13 12:15:50,431 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,431 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,431 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:50,431 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,431 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:50,445 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/31f78deccf794f5eaa06b39f4f050110 2023-02-13 12:15:50,451 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/bf44b77c2db742148b73ef8d840bb392 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392 2023-02-13 12:15:50,456 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:15:50,457 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/31f78deccf794f5eaa06b39f4f050110 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110 2023-02-13 12:15:50,461 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:15:50,462 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c234839e44de2733ae92a93b34e9e56e in 48ms, sequenceid=6, compaction requested=false 2023-02-13 12:15:50,462 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-13 12:15:50,463 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c234839e44de2733ae92a93b34e9e56e: 2023-02-13 12:15:50,463 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:50,463 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:50,463 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:50,463 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,463 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:50,463 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:50,473 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,473 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:50,473 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,473 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:50,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:50,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:50,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:50,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:50,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:50,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 12:15:50,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fca7996[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:50,475 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:15:50,475 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,484 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,484 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,484 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,484 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,484 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:50,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,484 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:50,484 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:50,484 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:50,484 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:50,484 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:50,484 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:50,484 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:50,484 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:50,485 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:50,492 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:50,492 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,492 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:50,492 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:50,500 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:50,500 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:50,500 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:50,509 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:50,509 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:50,509 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:50,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:50,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:50,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:50,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:15:50,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:15:50,513 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 12:15:50,513 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:50,513 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:50,513 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 12:15:50,513 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:50,522 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,522 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:50,523 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,523 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:50,523 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:50,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,523 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,523 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:50,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:50,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,524 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:50,525 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:50,525 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:50,526 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:50,528 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,528 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:50,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:50,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:50,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:50,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:50,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:50,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,534 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:50,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:50,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:50,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:50,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:50,534 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 12:15:50,534 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:50,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:50,534 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution because node does not exist (not an error) 2023-02-13 12:15:50,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:50,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:50,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:50,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:50,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:50,534 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:50,535 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:50,535 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:50,535 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:50,535 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:50,535 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:50,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:50,535 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,535 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,535 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:50,535 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,535 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,635 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:50,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:50,654 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:50,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:50,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 12:15:50,656 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 12:15:50,656 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:50,656 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 12:15:50,657 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:15:50,657 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,657 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,667 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:50,667 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,667 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:50,667 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:50,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:50,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:50,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:50,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:50,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:50,667 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:50,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:50,668 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:50,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,668 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,668 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,668 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:50,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:50,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:50,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:50,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:50,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:50,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:50,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:50,670 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:50,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:50,670 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:50,670 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:50,670 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:50,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:50,671 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:50,671 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:50,671 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:50,671 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:50,671 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:50,671 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:50,676 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:50,676 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:50,676 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:50,676 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:50,676 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:50,676 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:50,676 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:50,678 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:50,678 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:50,678 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:50,678 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. started... 2023-02-13 12:15:50,678 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c234839e44de2733ae92a93b34e9e56e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:50,681 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,689 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,689 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,689 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:50,690 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:50,690 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:50,690 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/b9228e02f0544f059205044d9f77d8a7 2023-02-13 12:15:51,106 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/f68d55b30ba94660b6013c606592d291 2023-02-13 12:15:51,115 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/b9228e02f0544f059205044d9f77d8a7 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7 2023-02-13 12:15:51,123 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:15:51,123 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/f68d55b30ba94660b6013c606592d291 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291 2023-02-13 12:15:51,128 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:15:51,129 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c234839e44de2733ae92a93b34e9e56e in 451ms, sequenceid=19, compaction requested=false 2023-02-13 12:15:51,130 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c234839e44de2733ae92a93b34e9e56e: 2023-02-13 12:15:51,130 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:51,130 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:51,130 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:51,130 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,130 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:51,130 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:51,139 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,139 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:51,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:51,140 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,140 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:51,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:51,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:51,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:51,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 12:15:51,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@452695d3[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:51,144 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:15:51,144 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,150 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,150 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,150 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,151 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:51,151 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:51,151 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:51,151 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,151 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:51,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,152 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:51,152 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:51,152 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:51,152 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:51,152 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:51,152 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:51,177 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:51,177 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,177 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:51,192 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:51,192 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:51,192 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,192 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:51,192 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,193 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:51,192 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:51,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:51,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:51,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:51,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:15:51,200 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:15:51,200 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 12:15:51,200 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:51,200 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:51,200 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 12:15:51,200 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:51,209 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,209 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:51,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:51,209 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,209 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,209 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,209 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:51,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,209 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,210 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,210 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:51,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,210 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:51,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,213 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,213 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,213 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,225 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,225 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,225 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,225 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,226 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,226 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,225 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,226 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,226 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 12:15:51,226 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:51,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,226 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,226 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:51,227 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:51,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,226 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,227 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,227 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,227 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,228 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,228 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,228 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,228 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,228 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,327 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:51,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:51,344 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:51,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:51,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 12:15:51,346 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 12:15:51,346 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,346 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 12:15:51,346 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:15:51,347 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,347 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,356 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,356 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,356 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,356 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,356 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,356 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:51,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,357 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,357 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,358 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:51,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:51,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,358 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:51,358 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:51,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:51,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,359 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:51,359 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:51,359 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:51,359 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:51,359 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:51,359 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:51,359 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,359 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:51,359 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:51,359 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:51,359 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:51,359 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:51,359 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:51,359 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:51,360 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,359 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,360 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:51,360 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:51,360 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:51,360 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:51,360 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:51,360 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:51,360 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:51,360 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. started... 2023-02-13 12:15:51,360 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c234839e44de2733ae92a93b34e9e56e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:51,367 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,376 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,376 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,376 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:51,376 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/380a4ee12e3947d9b3947fcd82806f94 2023-02-13 12:15:51,376 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,376 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:51,391 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/c2c41361412a4b44bae6b8ee8cbb16fa 2023-02-13 12:15:51,397 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/380a4ee12e3947d9b3947fcd82806f94 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94 2023-02-13 12:15:51,403 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:15:51,404 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/c2c41361412a4b44bae6b8ee8cbb16fa as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa 2023-02-13 12:15:51,409 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:15:51,410 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c234839e44de2733ae92a93b34e9e56e in 50ms, sequenceid=32, compaction requested=true 2023-02-13 12:15:51,410 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c234839e44de2733ae92a93b34e9e56e: 2023-02-13 12:15:51,410 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:51,410 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:51,410 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:51,410 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,410 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:51,410 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:51,417 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,417 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:51,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:51,417 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,418 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:51,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:51,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:51,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:51,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 12:15:51,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ece149a[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:51,419 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:15:51,419 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,425 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,425 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,425 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,426 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:51,426 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:51,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,426 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,426 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:51,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,426 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:51,426 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:51,426 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:51,426 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:51,426 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:51,426 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:51,426 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:51,434 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:51,434 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,434 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:51,442 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:51,442 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:51,442 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,442 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,442 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:51,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:51,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:51,442 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,442 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:51,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:51,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:51,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:51,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:15:51,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:15:51,446 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 12:15:51,446 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:51,446 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:51,446 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 12:15:51,446 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:51,456 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,456 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,456 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,456 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,456 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,456 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,456 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:51,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,457 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,456 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:51,457 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:51,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,459 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,459 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:51,460 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,495 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,495 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,495 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,495 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,495 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,495 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,495 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,495 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,496 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,495 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 12:15:51,496 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:51,495 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,496 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,496 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:51,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,495 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,497 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:51,496 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,597 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:51,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:51,617 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:51,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:51,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 12:15:51,619 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 12:15:51,619 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,619 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 12:15:51,619 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:15:51,619 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,619 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,625 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,625 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,625 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,625 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,626 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,626 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:51,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:51,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,627 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:51,627 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:51,627 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:51,627 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:51,627 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:51,627 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,627 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:51,627 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:51,627 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. started... 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:51,627 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c234839e44de2733ae92a93b34e9e56e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:51,627 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:51,639 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,643 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/9db6fc87a5544985a5b6944fcf2d00ba 2023-02-13 12:15:51,655 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2 2023-02-13 12:15:51,656 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,656 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,656 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:51,656 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,656 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:51,661 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/9db6fc87a5544985a5b6944fcf2d00ba as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba 2023-02-13 12:15:51,667 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:15:51,668 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2 2023-02-13 12:15:51,672 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:15:51,673 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c234839e44de2733ae92a93b34e9e56e in 46ms, sequenceid=45, compaction requested=true 2023-02-13 12:15:51,673 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c234839e44de2733ae92a93b34e9e56e: 2023-02-13 12:15:51,673 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:51,673 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:51,674 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:51,674 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,674 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:51,674 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:51,689 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,689 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:51,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:51,690 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,690 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:51,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:51,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:51,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:51,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 12:15:51,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@456c7306[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:51,692 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:15:51,693 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,706 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,706 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,706 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,707 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:51,707 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,707 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:51,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,707 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:51,707 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:51,707 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:51,707 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:51,707 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:51,707 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:51,708 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:51,708 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:51,723 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:51,723 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,723 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:51,731 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:51,731 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,731 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:51,736 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:51,736 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,736 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,736 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:51,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:51,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:51,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:51,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:51,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:51,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:15:51,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:15:51,741 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 12:15:51,741 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:51,741 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:51,741 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 12:15:51,741 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:51,750 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,750 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,750 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,751 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,750 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:51,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,750 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,751 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:51,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,751 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,751 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,752 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,758 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:51,758 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,760 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,769 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,769 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,769 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,769 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:51,769 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,770 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,770 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 12:15:51,770 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:51,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,769 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,769 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,770 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:51,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,770 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:51,770 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,770 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:51,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,770 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,771 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:51,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:51,771 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,771 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,771 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,771 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,771 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,771 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,870 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:51,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:51,887 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:51,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:51,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 12:15:51,890 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 12:15:51,890 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,890 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 12:15:51,890 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:15:51,890 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,890 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,893 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:15:51,900 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,900 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,900 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:51,900 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:51,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:51,901 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,901 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:51,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,901 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:51,902 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,902 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:51,902 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:51,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:51,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,902 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:51,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:51,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:51,903 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:51,903 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:51,903 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:51,903 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:51,903 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:51,903 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,903 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:51,903 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:51,904 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:51,905 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:51,905 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:51,905 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. started... 2023-02-13 12:15:51,905 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c234839e44de2733ae92a93b34e9e56e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:51,909 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,917 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,917 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:51,917 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,918 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,918 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:51,940 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/77d98e5dc5bd4e70998ca98cb12c43e8 2023-02-13 12:15:51,966 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/858978034d864591ba73ad8e21a94c91 2023-02-13 12:15:51,971 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/77d98e5dc5bd4e70998ca98cb12c43e8 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8 2023-02-13 12:15:51,976 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:15:51,978 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/858978034d864591ba73ad8e21a94c91 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91 2023-02-13 12:15:51,983 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:15:51,984 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c234839e44de2733ae92a93b34e9e56e in 79ms, sequenceid=58, compaction requested=true 2023-02-13 12:15:51,984 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c234839e44de2733ae92a93b34e9e56e: 2023-02-13 12:15:51,984 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:51,984 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:51,984 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:51,984 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,984 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:51,984 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:51,992 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,992 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:51,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:51,993 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:51,993 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:51,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:51,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:51,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:51,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:51,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:51,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:51,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:51,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 12:15:51,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1936a4f4[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:51,995 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:15:51,995 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,000 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,000 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,000 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,001 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:52,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,001 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,001 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:52,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,001 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:52,001 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:52,001 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:52,001 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:52,001 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:52,001 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:52,001 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:52,001 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:52,030 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:52,030 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:52,030 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:52,042 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:52,042 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:52,043 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:52,042 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,042 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:52,043 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:52,043 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:52,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:52,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:52,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:52,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:15:52,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:15:52,054 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 12:15:52,054 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:52,054 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:52,054 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 12:15:52,054 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:52,064 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,064 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,064 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,064 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,065 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:52,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,066 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,065 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,064 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,066 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,067 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:52,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,072 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,073 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:52,073 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,083 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 12:15:52,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:52,084 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:52,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:52,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:52,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:52,084 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,085 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,085 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,085 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,085 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,085 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,184 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:52,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:52,205 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:15:52,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 12:15:52,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 12:15:52,207 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 12:15:52,207 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:52,207 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 12:15:52,207 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:15:52,208 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,208 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,217 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:52,217 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,217 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:52,217 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:52,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:52,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:52,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:52,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:52,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:52,218 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,218 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:15:52,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:52,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,218 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,218 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,219 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:52,219 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 12:15:52,220 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:52,219 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:52,220 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:52,220 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:15:52,220 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:52,221 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:52,221 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:15:52,221 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:15:52,221 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:52,221 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. started... 2023-02-13 12:15:52,221 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c234839e44de2733ae92a93b34e9e56e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:15:52,231 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,234 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/a0c4e5b53f5344abb303f35b8e985267 2023-02-13 12:15:52,239 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,239 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:52,239 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,240 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,240 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:52,256 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/52da198e2e5840eb8e6d2c000f69ae6d 2023-02-13 12:15:52,261 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/a0c4e5b53f5344abb303f35b8e985267 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267 2023-02-13 12:15:52,266 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:15:52,267 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/52da198e2e5840eb8e6d2c000f69ae6d as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d 2023-02-13 12:15:52,272 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:15:52,273 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c234839e44de2733ae92a93b34e9e56e in 52ms, sequenceid=71, compaction requested=true 2023-02-13 12:15:52,273 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c234839e44de2733ae92a93b34e9e56e: 2023-02-13 12:15:52,273 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:52,273 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:15:52,273 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:15:52,273 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,273 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 12:15:52,273 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 12:15:52,284 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,284 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:52,284 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,284 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:15:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:52,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 12:15:52,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68cf320c[Count = 0] remaining members to acquire global barrier 2023-02-13 12:15:52,286 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:15:52,286 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,292 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,292 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,292 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,292 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,292 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:15:52,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,292 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,293 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:52,293 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:15:52,293 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:52,293 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:52,293 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:15:52,293 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 12:15:52,293 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:15:52,293 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:15:52,300 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,300 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:52,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:52,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:52,301 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:52,301 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:52,309 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:52,309 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 12:15:52,309 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:52,309 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:52,309 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:52,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:52,309 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 12:15:52,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:52,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:52,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:15:52,312 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:15:52,312 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 12:15:52,312 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:15:52,312 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:15:52,312 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 12:15:52,312 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:15:52,322 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,323 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,323 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,322 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,323 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,323 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,323 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:15:52,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:15:52,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:15:52,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,326 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:15:52,326 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,327 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:15:52,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 12:15:52,328 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,328 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,329 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,364 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,364 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:52,364 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,364 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:15:52,364 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:52,364 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:15:52,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:52,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:52,364 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,364 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 12:15:52,365 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:15:52,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:52,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:52,364 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:15:52,364 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:15:52,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 12:15:52,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:15:52,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:15:52,365 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 12:15:52,465 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:15:52,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:15:52,733 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 12:15:52,734 DEBUG [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:52,736 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,737 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,737 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,737 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,738 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:52,739 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:52,739 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:52,986 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,986 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:52,986 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:52,986 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:52,986 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-13 12:15:52,986 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:52,987 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:52,987 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:52,987 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:52,987 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:52,988 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 12:15:52,989 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:52,989 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:52,989 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:52,989 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:52,990 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:52,990 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:52,990 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:52,990 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:52,991 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:52,991 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:52,991 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:52,991 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:52,991 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:52,992 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,992 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:52,992 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,992 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,992 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,992 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 12:15:52,993 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:52,993 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:52,994 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290582994 2023-02-13 12:15:52,995 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:52,995 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:52,998 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:52,998 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:52,998 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:52,999 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290612998 2023-02-13 12:15:52,999 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290672999 2023-02-13 12:15:52,999 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:52,999 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:53,001 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:53,001 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:53,001 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:53,249 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,249 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:53,249 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:53,250 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:53,250 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:53,250 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:53,251 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:53,251 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:53,251 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:53,252 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:53,255 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 12:15:53,255 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:53,256 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:53,257 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:53,257 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:53,257 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:53,258 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:53,258 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:53,258 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:53,258 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:53,259 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:53,259 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:53,259 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:53,259 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:53,259 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,260 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:53,260 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,260 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,260 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,260 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:15:53,261 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:53,261 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:53,261 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290583261 2023-02-13 12:15:53,263 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:53,263 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,265 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:53,265 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:53,265 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:53,265 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290613265 2023-02-13 12:15:53,265 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290673265 2023-02-13 12:15:53,265 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 12:15:53,266 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:53,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN}] 2023-02-13 12:15:53,268 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:53,268 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:53,268 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:53,516 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,516 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:53,516 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:53,517 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:53,517 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:53,518 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:53,518 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:53,518 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:53,518 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:53,521 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 2 msec 2023-02-13 12:15:53,526 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 12:15:53,527 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:53,528 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:53,528 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:53,528 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:53,529 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:53,529 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:53,529 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:53,530 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:53,530 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:53,531 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:53,531 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:53,532 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:53,532 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:53,533 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,533 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:53,533 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,533 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,533 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,533 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-13 12:15:53,534 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:53,534 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:53,535 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290583535 2023-02-13 12:15:53,535 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:53,536 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,538 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:53,538 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:53,538 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:53,538 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290613538 2023-02-13 12:15:53,538 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290673538 2023-02-13 12:15:53,539 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:53,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN}] 2023-02-13 12:15:53,539 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:53,541 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:53,542 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:53,543 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290553542"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290553542"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290553542"}]},"ts":"1676290553542"} 2023-02-13 12:15:53,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:15:53,544 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:53,544 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:53,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:53,545 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:53,789 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,789 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:53,789 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:53,789 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:53,790 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:53,790 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:53,790 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:53,790 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:53,790 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:53,791 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:53,793 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 12:15:53,793 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 12:15:53,793 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:53,793 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:53,794 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:53,794 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:53,794 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:53,795 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:53,795 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:53,795 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:53,795 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:53,795 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:53,796 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:53,796 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:53,796 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:53,797 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,797 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:53,797 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:53,797 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,797 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,797 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:53,797 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:53,797 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:53,797 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 12:15:53,798 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:53,798 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:53,799 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e to restore RIT 2023-02-13 12:15:53,799 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290583799 2023-02-13 12:15:53,800 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:53,800 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:53,802 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:53,802 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSING, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:53,802 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:53,802 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290613802 2023-02-13 12:15:53,802 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290673802 2023-02-13 12:15:53,802 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 12:15:53,802 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:53,803 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290553802"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290553802"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290553802"}]},"ts":"1676290553802"} 2023-02-13 12:15:53,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:15:53,804 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:53,958 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:53,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing c234839e44de2733ae92a93b34e9e56e, disabling compactions & flushes 2023-02-13 12:15:53,958 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:53,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:53,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. after waiting 0 ms 2023-02-13 12:15:53,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:53,958 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing c234839e44de2733ae92a93b34e9e56e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 12:15:53,969 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/e9fa53d9a2c047448944f6246c1bc6d7 2023-02-13 12:15:53,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/d1a2d45c330f4009adc456370b54f98b 2023-02-13 12:15:53,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/e9fa53d9a2c047448944f6246c1bc6d7 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7 2023-02-13 12:15:53,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:15:54,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/d1a2d45c330f4009adc456370b54f98b as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b 2023-02-13 12:15:54,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:15:54,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c234839e44de2733ae92a93b34e9e56e in 49ms, sequenceid=82, compaction requested=true 2023-02-13 12:15:54,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 12:15:54,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:15:54,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:54,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for c234839e44de2733ae92a93b34e9e56e: Waiting for close lock at 1676290553958Running coprocessor pre-close hooks at 1676290553958Disabling compacts and flushes for region at 1676290553958Disabling writes for close at 1676290553958Obtaining lock to block concurrent updates at 1676290553958Preparing flush snapshotting stores in c234839e44de2733ae92a93b34e9e56e at 1676290553958Finished memstore snapshotting testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676290553959 (+1 ms)Flushing stores of testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. at 1676290553959Flushing c234839e44de2733ae92a93b34e9e56e/cf1: creating writer at 1676290553960 (+1 ms)Flushing c234839e44de2733ae92a93b34e9e56e/cf1: appending metadata at 1676290553962 (+2 ms)Flushing c234839e44de2733ae92a93b34e9e56e/cf1: closing flushed file at 1676290553962Flushing c234839e44de2733ae92a93b34e9e56e/cf2: creating writer at 1676290553975 (+13 ms)Flushing c234839e44de2733ae92a93b34e9e56e/cf2: appending metadata at 1676290553978 (+3 ms)Flushing c234839e44de2733ae92a93b34e9e56e/cf2: closing flushed file at 1676290553978Flushing c234839e44de2733ae92a93b34e9e56e/cf1: reopening flushed file at 1676290553994 (+16 ms)Flushing c234839e44de2733ae92a93b34e9e56e/cf2: reopening flushed file at 1676290554001 (+7 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c234839e44de2733ae92a93b34e9e56e in 49ms, sequenceid=82, compaction requested=true at 1676290554007 (+6 ms)Writing region close event to WAL at 1676290554008 (+1 ms)Running coprocessor post-close hooks at 1676290554013 (+5 ms)Closed at 1676290554014 (+1 ms) 2023-02-13 12:15:54,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,017 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED 2023-02-13 12:15:54,018 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290554017"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290554017"}]},"ts":"1676290554017"} 2023-02-13 12:15:54,019 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:54,020 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:54,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:54,020 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:54,052 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,053 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:54,053 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:54,053 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:54,054 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:54,054 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:54,054 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:54,054 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:54,055 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:54,056 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:54,058 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 12:15:54,058 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 12:15:54,058 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 12:15:54,058 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:54,059 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:54,059 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:54,060 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:54,060 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:54,060 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:54,060 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:54,061 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:54,061 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:54,061 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:54,061 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:54,062 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:54,062 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:54,062 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,063 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:54,063 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:54,063 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:54,063 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,063 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,063 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,063 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:54,063 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:54,063 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:54,063 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:15:54,064 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:54,065 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:54,065 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e to restore RIT 2023-02-13 12:15:54,065 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290584065 2023-02-13 12:15:54,066 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:54,066 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:54,069 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:54,069 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:54,069 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:54,070 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290614070 2023-02-13 12:15:54,070 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290674070 2023-02-13 12:15:54,070 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:54,070 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:54,070 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290554070"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290554070"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290554070"}]},"ts":"1676290554070"} 2023-02-13 12:15:54,072 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:54,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-13 12:15:54,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 in 268 msec 2023-02-13 12:15:54,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:54,075 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:54,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:54,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:54,320 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,320 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:54,321 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:54,321 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:54,321 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:54,321 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:54,322 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:54,322 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:54,322 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:54,323 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:54,325 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 12:15:54,326 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 12:15:54,326 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:54,327 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:54,328 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:54,328 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:54,329 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:54,329 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:54,330 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:54,330 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:54,331 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:54,331 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:54,332 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:54,332 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:54,332 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:54,332 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:54,333 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,333 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:54,333 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:54,333 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:54,334 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,334 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,334 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,334 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:54,334 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:54,334 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 12:15:54,335 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:54,335 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:54,335 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e to restore RIT 2023-02-13 12:15:54,335 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290584335 2023-02-13 12:15:54,336 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:54,337 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:54,339 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:54,339 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:54,339 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:54,339 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290614339 2023-02-13 12:15:54,339 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290674339 2023-02-13 12:15:54,339 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:54,339 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:54,341 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:54,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-13 12:15:54,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN in 802 msec 2023-02-13 12:15:54,349 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:54,349 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:54,349 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:54,589 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,590 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:54,590 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:54,590 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:54,590 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:54,591 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:54,591 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:54,591 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:54,591 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:54,593 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:54,595 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 12:15:54,596 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:54,596 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:54,596 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:54,596 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:54,597 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:54,597 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:54,597 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:54,597 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:54,598 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:54,598 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:54,598 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:54,599 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:54,599 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:54,599 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:54,599 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:54,600 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,600 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:54,600 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:54,600 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:54,600 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:54,600 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:54,601 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:15:54,601 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:54,602 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:54,602 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290584602 2023-02-13 12:15:54,604 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,604 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:54,606 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:54,606 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:54,607 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:54,607 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290614607 2023-02-13 12:15:54,607 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290674607 2023-02-13 12:15:54,607 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:54,607 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,607 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,607 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:54,613 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:54,622 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=c234839e44de2733ae92a93b34e9e56e, threads=14 2023-02-13 12:15:54,623 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,623 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,623 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,623 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,623 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,623 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,624 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,624 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,624 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,624 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,624 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,624 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,624 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,625 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,790 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,796 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,796 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,804 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,809 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,825 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,825 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,826 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,840 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,843 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,843 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,846 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:54,846 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,184 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,185 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region c234839e44de2733ae92a93b34e9e56e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 12:15:55,225 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:55,225 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:55,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:55,225 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:55,357 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:55,358 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:55,358 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:55,358 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:55,363 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:55,364 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:55,364 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:55,364 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:55,364 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:55,367 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 2 msec 2023-02-13 12:15:55,370 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 12:15:55,370 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:55,371 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:55,371 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:55,371 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:55,372 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:55,372 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:55,372 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:55,372 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:55,372 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:55,372 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:55,373 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:55,373 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:55,373 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:55,373 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:55,373 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:55,374 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:55,374 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:55,374 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:55,374 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:55,374 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:55,374 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:55,374 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:55,374 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:55,374 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:55,374 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:15:55,375 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:55,376 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:55,376 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290585376 2023-02-13 12:15:55,377 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:55,377 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:55,379 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:55,379 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:55,379 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:55,379 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290615379 2023-02-13 12:15:55,379 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290675379 2023-02-13 12:15:55,379 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 12:15:55,388 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=c234839e44de2733ae92a93b34e9e56e, threads=14 2023-02-13 12:15:55,389 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,389 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,389 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,389 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,389 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,390 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,390 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,390 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,390 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,390 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,391 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,391 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,391 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,391 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,528 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,530 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,551 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,557 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,558 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,560 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,564 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,564 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,566 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,569 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,572 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,572 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,576 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,880 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2 for region: c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:55,880 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region c234839e44de2733ae92a93b34e9e56e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 12:15:55,947 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:55,952 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/5dc4f3767bed16f7e8e012be0cd512b5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:15:55,953 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/5188b35eb20e08a4c9da35109d932d6e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:15:55,953 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:55,953 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:55,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:55,954 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:56,130 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,130 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:56,130 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:56,130 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:56,131 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:56,131 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:56,131 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:56,131 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:56,131 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:56,133 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:56,134 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 12:15:56,135 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:56,135 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:56,135 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:56,136 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:56,136 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:56,137 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:56,137 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,138 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:56,138 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,138 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,139 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,140 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:56,140 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,140 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,140 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:56,141 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,141 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:56,141 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:56,143 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:56,143 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,143 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,144 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,144 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:56,144 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:56,144 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 12:15:56,144 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:56,145 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:56,147 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290586147 2023-02-13 12:15:56,149 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,149 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:56,152 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:56,152 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:56,152 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:56,153 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290616152 2023-02-13 12:15:56,153 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290676153 2023-02-13 12:15:56,153 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:56,156 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:56,160 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:56,160 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:56,160 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:56,404 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,404 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:56,404 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:56,404 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:56,405 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:56,405 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:56,405 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:56,405 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:56,406 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:56,406 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:56,407 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 12:15:56,408 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:56,408 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:56,408 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:56,408 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:56,408 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:56,409 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:56,409 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,409 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:56,410 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,410 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,410 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,410 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:56,410 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,411 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,411 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:56,411 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,411 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:56,411 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:56,412 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:56,412 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,412 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,412 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,412 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:56,412 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:56,412 DEBUG [Listener at localhost.localdomain/46087] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,412 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 12:15:56,416 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:56,417 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:56,419 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290586419 2023-02-13 12:15:56,422 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,422 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:56,424 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:56,425 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:56,425 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:56,425 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290616425 2023-02-13 12:15:56,425 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290676425 2023-02-13 12:15:56,425 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:56,426 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:56,438 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:56,438 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 12:15:56,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:56,438 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:56,677 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,677 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:56,677 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:56,677 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:56,677 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:56,677 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:56,678 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:56,678 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:56,678 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:56,678 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:56,679 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 12:15:56,680 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:56,680 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:56,680 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:56,680 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:56,680 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:56,681 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:56,681 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,681 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:56,681 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,681 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,682 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,682 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:56,682 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,682 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,682 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:56,683 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,683 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:56,683 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:56,683 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:56,684 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,684 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,684 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,684 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:56,684 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:56,684 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 12:15:56,684 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:56,685 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:56,685 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290586685 2023-02-13 12:15:56,688 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:56,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,691 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:56,691 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:56,691 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:56,691 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290616691 2023-02-13 12:15:56,691 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290676691 2023-02-13 12:15:56,691 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 12:15:56,691 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:56,695 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:56,696 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 12:15:56,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:56,696 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:56,942 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,942 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:56,943 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:56,943 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:56,943 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:56,943 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:56,943 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:56,943 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:56,944 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:56,945 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:56,946 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 12:15:56,946 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:56,946 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:56,947 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:56,947 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:56,948 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:56,948 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:56,948 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,949 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:56,949 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,949 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:56,950 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,950 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:56,951 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,951 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:56,951 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:56,953 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,953 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:56,953 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:56,953 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:56,953 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,953 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,954 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,954 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:56,954 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:56,954 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 12:15:56,955 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:56,955 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:56,956 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290586956 2023-02-13 12:15:56,956 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:56,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:56,959 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:56,959 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:56,959 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:56,960 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290616960 2023-02-13 12:15:56,960 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290676960 2023-02-13 12:15:56,960 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:56,960 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:56,962 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:56,962 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 12:15:56,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:56,963 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:57,210 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,210 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:57,210 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:57,211 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:57,212 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:57,213 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:57,213 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:57,213 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:57,213 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:57,214 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:57,215 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 12:15:57,215 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:57,216 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:57,216 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:57,217 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:57,217 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:57,217 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:57,218 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:57,218 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:57,218 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:57,219 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:57,219 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:57,219 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:57,219 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:57,220 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:57,220 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:57,220 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,220 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:57,221 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:57,221 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:57,221 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,221 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,221 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,221 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:57,221 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:57,221 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:15:57,222 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:57,223 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:57,223 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290587223 2023-02-13 12:15:57,225 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:57,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,228 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:57,228 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:57,228 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:57,229 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290617228 2023-02-13 12:15:57,229 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290677229 2023-02-13 12:15:57,229 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:57,229 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:57,235 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN] 2023-02-13 12:15:57,235 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:57,235 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 12:15:57,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:57,235 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:57,479 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,479 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:57,479 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:57,479 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:57,480 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:57,480 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:57,480 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:57,480 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:57,480 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:57,481 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:57,482 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 12:15:57,483 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:15:57,483 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:15:57,483 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 12:15:57,483 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:57,483 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:57,484 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:57,484 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:57,484 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:57,485 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:57,485 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:57,485 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:57,485 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:57,486 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:57,486 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:57,486 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:57,486 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:57,487 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,488 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:57,488 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:57,488 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:57,488 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:57,488 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,488 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,488 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,488 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:57,488 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:57,488 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:57,489 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:15:57,489 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:57,490 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:57,490 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e to restore RIT 2023-02-13 12:15:57,490 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290587490 2023-02-13 12:15:57,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,492 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:57,493 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:57,495 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:57,495 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:57,495 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:57,495 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290617495 2023-02-13 12:15:57,495 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290677495 2023-02-13 12:15:57,495 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:57,495 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:57,501 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:57,501 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 12:15:57,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:57,502 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:57,745 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,746 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:57,746 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:57,746 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:57,747 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:57,747 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:57,747 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:57,747 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:57,748 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:57,749 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:57,752 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 12:15:57,752 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 12:15:57,752 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:57,753 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:57,754 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:57,754 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:57,755 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:57,756 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:57,756 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:57,756 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:57,756 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:57,756 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:57,756 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:57,756 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:57,757 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:57,757 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,757 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:57,757 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:57,757 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,757 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,757 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,757 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:57,757 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:15:57,758 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:57,759 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:57,759 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e to restore RIT 2023-02-13 12:15:57,759 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290587759 2023-02-13 12:15:57,760 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:57,761 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:57,762 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:57,763 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:57,764 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:57,764 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:57,764 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290617764 2023-02-13 12:15:57,764 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290677764 2023-02-13 12:15:57,764 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:57,764 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:57,766 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:57,766 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 12:15:57,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:57,766 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:58,014 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,014 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:58,014 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:58,015 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:58,015 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:58,015 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:58,015 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:58,015 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:58,016 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:58,017 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:58,018 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 12:15:58,019 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 12:15:58,019 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:58,019 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:58,020 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:58,020 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:58,021 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:58,021 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:58,022 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:58,022 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:58,023 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:58,023 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:58,023 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:58,023 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:58,024 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:58,025 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,025 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:58,025 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:58,025 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,025 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,025 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,026 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:58,026 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 12:15:58,027 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:58,028 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:58,028 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e to restore RIT 2023-02-13 12:15:58,029 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290588029 2023-02-13 12:15:58,030 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:58,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,031 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:58,033 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:58,033 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:58,033 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:58,033 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290618033 2023-02-13 12:15:58,033 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290678033 2023-02-13 12:15:58,033 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:15:58,033 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:58,035 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:58,035 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 12:15:58,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:58,035 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:58,283 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,283 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:58,284 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:58,284 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:58,285 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:58,285 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:58,285 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:58,285 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:58,285 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:58,286 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:15:58,287 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 12:15:58,287 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 12:15:58,288 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:58,288 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:58,288 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:58,288 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:58,289 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:58,289 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:58,289 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:58,290 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:58,290 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:58,290 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:58,291 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:58,291 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:58,292 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:58,292 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,292 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:58,293 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:58,293 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,293 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,293 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,293 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:15:58,293 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:15:58,294 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:58,295 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:58,295 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e to restore RIT 2023-02-13 12:15:58,295 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290588295 2023-02-13 12:15:58,297 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,297 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:58,300 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:58,302 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:58,302 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:58,302 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:58,303 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290618303 2023-02-13 12:15:58,303 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290678303 2023-02-13 12:15:58,303 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 12:15:58,303 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:15:58,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e exec-time=5.5690 sec 2023-02-13 12:15:58,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:58,307 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN; state=CLOSED, location=jenkins-hbase16.apache.org,33961,1676290533759; forceNewPlan=false, retain=true 2023-02-13 12:15:58,457 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:15:58,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:58,459 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290558458"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290558458"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290558458"}]},"ts":"1676290558458"} 2023-02-13 12:15:58,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:15:58,461 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:15:58,461 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:58,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:15:58,462 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:15:58,553 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:15:58,553 WARN [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-13 12:15:58,553 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:15:58,553 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:15:58,554 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:15:58,554 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:15:58,554 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:15:58,554 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:15:58,554 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:15:58,555 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:15:58,556 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:15:58,558 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-13 12:15:58,558 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:15:58,559 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:15:58,559 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:15:58,559 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:15:58,560 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:58,560 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:15:58,560 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:58,561 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:15:58,561 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:58,561 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:15:58,561 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:58,561 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:15:58,562 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:58,562 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:15:58,562 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:58,562 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:15:58,562 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:58,562 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 12:15:58,563 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:15:58,564 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:15:58,564 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e to restore RIT 2023-02-13 12:15:58,564 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290588564 2023-02-13 12:15:58,565 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:15:58,565 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:15:58,567 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:15:58,567 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=c234839e44de2733ae92a93b34e9e56e, regionState=OPENING, lastHost=jenkins-hbase16.apache.org,33961,1676290533759, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759, openSeqNum=2 2023-02-13 12:15:58,567 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:15:58,568 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290618568 2023-02-13 12:15:58,568 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290678568 2023-02-13 12:15:58,568 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 12:15:58,568 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:58,568 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290558568"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290558568"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290558568"}]},"ts":"1676290558568"} 2023-02-13 12:15:58,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:15:58,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:58,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c234839e44de2733ae92a93b34e9e56e, NAME => 'testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:15:58,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:58,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:58,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:58,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:58,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:58,727 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:58,729 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1 2023-02-13 12:15:58,729 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1 2023-02-13 12:15:58,729 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c234839e44de2733ae92a93b34e9e56e columnFamilyName cf1 2023-02-13 12:15:58,740 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94 2023-02-13 12:15:58,746 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8 2023-02-13 12:15:58,752 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba 2023-02-13 12:15:58,758 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267 2023-02-13 12:15:58,762 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7 2023-02-13 12:15:58,765 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392 2023-02-13 12:15:58,769 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7 2023-02-13 12:15:58,769 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(310): Store=c234839e44de2733ae92a93b34e9e56e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:58,770 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:58,770 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2 2023-02-13 12:15:58,770 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2 2023-02-13 12:15:58,771 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c234839e44de2733ae92a93b34e9e56e columnFamilyName cf2 2023-02-13 12:15:58,778 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110 2023-02-13 12:15:58,782 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d 2023-02-13 12:15:58,787 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91 2023-02-13 12:15:58,791 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2 2023-02-13 12:15:58,794 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa 2023-02-13 12:15:58,798 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b 2023-02-13 12:15:58,801 DEBUG [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291 2023-02-13 12:15:58,801 INFO [StoreOpener-c234839e44de2733ae92a93b34e9e56e-1] regionserver.HStore(310): Store=c234839e44de2733ae92a93b34e9e56e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:58,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:58,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:58,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.splits 2023-02-13 12:15:58,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:15:58,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:58,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened c234839e44de2733ae92a93b34e9e56e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71589535, jitterRate=0.06676720082759857}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:15:58,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for c234839e44de2733ae92a93b34e9e56e: Running coprocessor pre-open hook at 1676290558726Writing region info on filesystem at 1676290558726Initializing all the Stores at 1676290558727 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290558727Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290558727Cleaning up temporary data from old regions at 1676290558803 (+76 ms)Cleaning up detritus from prior splits at 1676290558804 (+1 ms)Running coprocessor post-open hooks at 1676290558807 (+3 ms)Region opened successfully at 1676290558807 2023-02-13 12:15:58,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e., pid=48, masterSystemTime=1676290558721 2023-02-13 12:15:58,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:58,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:15:58,809 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:15:58,809 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:15:58,810 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 12:15:58,810 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 12:15:58,811 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1912): c234839e44de2733ae92a93b34e9e56e/cf2 is initiating minor compaction (all files) 2023-02-13 12:15:58,811 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1912): c234839e44de2733ae92a93b34e9e56e/cf1 is initiating minor compaction (all files) 2023-02-13 12:15:58,811 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c234839e44de2733ae92a93b34e9e56e/cf2 in testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:58,811 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c234839e44de2733ae92a93b34e9e56e/cf1 in testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:58,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:58,811 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp, totalSize=34.9 K 2023-02-13 12:15:58,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:58,811 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp, totalSize=34.9 K 2023-02-13 12:15:58,812 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:58,812 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 31f78deccf794f5eaa06b39f4f050110, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290550392 2023-02-13 12:15:58,812 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting bf44b77c2db742148b73ef8d840bb392, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290550392 2023-02-13 12:15:58,812 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290558812"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290558812"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290558812"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290558812"}]},"ts":"1676290558812"} 2023-02-13 12:15:58,813 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting f68d55b30ba94660b6013c606592d291, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290550637 2023-02-13 12:15:58,813 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting b9228e02f0544f059205044d9f77d8a7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290550637 2023-02-13 12:15:58,813 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting c2c41361412a4b44bae6b8ee8cbb16fa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290551329 2023-02-13 12:15:58,813 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting 380a4ee12e3947d9b3947fcd82806f94, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290551329 2023-02-13 12:15:58,813 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting bff4b7e9b43941b9ba4bdcf54c709eb2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290551599 2023-02-13 12:15:58,814 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting 9db6fc87a5544985a5b6944fcf2d00ba, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290551599 2023-02-13 12:15:58,814 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 858978034d864591ba73ad8e21a94c91, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290551872 2023-02-13 12:15:58,814 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting 77d98e5dc5bd4e70998ca98cb12c43e8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290551872 2023-02-13 12:15:58,814 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 52da198e2e5840eb8e6d2c000f69ae6d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290552187 2023-02-13 12:15:58,815 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting a0c4e5b53f5344abb303f35b8e985267, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290552187 2023-02-13 12:15:58,815 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting d1a2d45c330f4009adc456370b54f98b, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290552469 2023-02-13 12:15:58,815 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting e9fa53d9a2c047448944f6246c1bc6d7, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290552469 2023-02-13 12:15:58,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-13 12:15:58,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 in 245 msec 2023-02-13 12:15:58,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN in 1.5840 sec 2023-02-13 12:15:58,819 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-13 12:15:58,821 DEBUG [Listener at localhost.localdomain/46087] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-13 12:15:58,823 INFO [Listener at localhost.localdomain/46087] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 12:15:58,825 INFO [Listener at localhost.localdomain/46087] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-13 12:15:58,826 INFO [Listener at localhost.localdomain/46087] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 12:15:58,826 INFO [Listener at localhost.localdomain/46087] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-13 12:15:58,839 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] throttle.PressureAwareThroughputController(145): c234839e44de2733ae92a93b34e9e56e#cf2#compaction#50 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:15:58,844 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] throttle.PressureAwareThroughputController(145): c234839e44de2733ae92a93b34e9e56e#cf1#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:15:58,882 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf2/199bb3efd72f4a3e9a2ee3f4d872ce9f as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/199bb3efd72f4a3e9a2ee3f4d872ce9f 2023-02-13 12:15:58,883 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/.tmp/cf1/939923b3eaaf42f19ec547544b39849e as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/939923b3eaaf42f19ec547544b39849e 2023-02-13 12:15:58,892 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c234839e44de2733ae92a93b34e9e56e/cf2 of c234839e44de2733ae92a93b34e9e56e into 199bb3efd72f4a3e9a2ee3f4d872ce9f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:15:58,892 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c234839e44de2733ae92a93b34e9e56e: 2023-02-13 12:15:58,892 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e., storeName=c234839e44de2733ae92a93b34e9e56e/cf2, priority=9, startTime=1676290558808; duration=0sec 2023-02-13 12:15:58,893 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c234839e44de2733ae92a93b34e9e56e/cf1 of c234839e44de2733ae92a93b34e9e56e into 939923b3eaaf42f19ec547544b39849e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:15:58,894 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:58,894 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c234839e44de2733ae92a93b34e9e56e: 2023-02-13 12:15:58,894 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e., storeName=c234839e44de2733ae92a93b34e9e56e/cf1, priority=9, startTime=1676290558808; duration=0sec 2023-02-13 12:15:58,894 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:15:58,906 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:15:58,908 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-13 12:15:58,908 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testRollbackAndDoubleExecution 2023-02-13 12:15:58,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:58,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 12:15:58,912 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290558912"}]},"ts":"1676290558912"} 2023-02-13 12:15:58,913 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 12:15:58,933 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-13 12:15:58,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN}] 2023-02-13 12:15:58,935 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN 2023-02-13 12:15:58,936 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:15:58,936 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290558936"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290558936"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290558936"}]},"ts":"1676290558936"} 2023-02-13 12:15:58,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:15:59,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 12:15:59,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:59,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing c234839e44de2733ae92a93b34e9e56e, disabling compactions & flushes 2023-02-13 12:15:59,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:59,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:59,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. after waiting 0 ms 2023-02-13 12:15:59,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:59,092 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7] to archive 2023-02-13 12:15:59,093 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:15:59,097 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/bf44b77c2db742148b73ef8d840bb392 2023-02-13 12:15:59,100 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/b9228e02f0544f059205044d9f77d8a7 2023-02-13 12:15:59,102 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/380a4ee12e3947d9b3947fcd82806f94 2023-02-13 12:15:59,103 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/9db6fc87a5544985a5b6944fcf2d00ba 2023-02-13 12:15:59,104 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/77d98e5dc5bd4e70998ca98cb12c43e8 2023-02-13 12:15:59,106 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/a0c4e5b53f5344abb303f35b8e985267 2023-02-13 12:15:59,107 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/e9fa53d9a2c047448944f6246c1bc6d7 2023-02-13 12:15:59,111 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b] to archive 2023-02-13 12:15:59,112 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:15:59,113 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/31f78deccf794f5eaa06b39f4f050110 2023-02-13 12:15:59,114 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/f68d55b30ba94660b6013c606592d291 2023-02-13 12:15:59,115 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/c2c41361412a4b44bae6b8ee8cbb16fa 2023-02-13 12:15:59,116 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/bff4b7e9b43941b9ba4bdcf54c709eb2 2023-02-13 12:15:59,117 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/858978034d864591ba73ad8e21a94c91 2023-02-13 12:15:59,117 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/52da198e2e5840eb8e6d2c000f69ae6d 2023-02-13 12:15:59,118 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/d1a2d45c330f4009adc456370b54f98b 2023-02-13 12:15:59,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 12:15:59,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:15:59,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e. 2023-02-13 12:15:59,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for c234839e44de2733ae92a93b34e9e56e: Waiting for close lock at 1676290559091Running coprocessor pre-close hooks at 1676290559091Disabling compacts and flushes for region at 1676290559091Disabling writes for close at 1676290559091Writing region close event to WAL at 1676290559119 (+28 ms)Running coprocessor post-close hooks at 1676290559122 (+3 ms)Closed at 1676290559122 2023-02-13 12:15:59,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:59,124 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=c234839e44de2733ae92a93b34e9e56e, regionState=CLOSED 2023-02-13 12:15:59,125 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290559124"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290559124"}]},"ts":"1676290559124"} 2023-02-13 12:15:59,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-13 12:15:59,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure c234839e44de2733ae92a93b34e9e56e, server=jenkins-hbase16.apache.org,33961,1676290533759 in 188 msec 2023-02-13 12:15:59,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-13 12:15:59,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, UNASSIGN in 194 msec 2023-02-13 12:15:59,130 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290559130"}]},"ts":"1676290559130"} 2023-02-13 12:15:59,131 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 12:15:59,164 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-13 12:15:59,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 257 msec 2023-02-13 12:15:59,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 12:15:59,214 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-13 12:15:59,215 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testRollbackAndDoubleExecution 2023-02-13 12:15:59,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:59,217 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:59,218 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:59,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 12:15:59,221 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:59,223 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/recovered.edits] 2023-02-13 12:15:59,226 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/939923b3eaaf42f19ec547544b39849e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf1/939923b3eaaf42f19ec547544b39849e 2023-02-13 12:15:59,228 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/199bb3efd72f4a3e9a2ee3f4d872ce9f to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/cf2/199bb3efd72f4a3e9a2ee3f4d872ce9f 2023-02-13 12:15:59,230 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/recovered.edits/90.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e/recovered.edits/90.seqid 2023-02-13 12:15:59,231 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackAndDoubleExecution/c234839e44de2733ae92a93b34e9e56e 2023-02-13 12:15:59,231 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 12:15:59,233 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:59,235 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-13 12:15:59,237 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-13 12:15:59,238 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:59,238 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-13 12:15:59,238 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290559238"}]},"ts":"9223372036854775807"} 2023-02-13 12:15:59,240 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 12:15:59,240 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c234839e44de2733ae92a93b34e9e56e, NAME => 'testRollbackAndDoubleExecution,,1676290549637.c234839e44de2733ae92a93b34e9e56e.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 12:15:59,240 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-13 12:15:59,240 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676290559240"}]},"ts":"9223372036854775807"} 2023-02-13 12:15:59,242 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-13 12:15:59,251 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:15:59,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 36 msec 2023-02-13 12:15:59,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 12:15:59,320 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-13 12:15:59,334 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=402 (was 404), OpenFileDescriptor=847 (was 876), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=321 (was 260) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=5432 (was 5639) 2023-02-13 12:15:59,345 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=402, OpenFileDescriptor=847, MaxFileDescriptor=60000, SystemLoadAverage=321, ProcessCount=167, AvailableMemoryMB=5431 2023-02-13 12:15:59,345 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:15:59,378 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 12:15:59,379 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 12:15:59,380 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:15:59,382 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:15:59,382 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:15:59,384 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302 2023-02-13 12:15:59,384 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302 empty. 2023-02-13 12:15:59,385 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302 2023-02-13 12:15:59,385 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 12:15:59,394 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-13 12:15:59,395 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 47636dde92dbddaf4a0d879337d26302, NAME => 'testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:15:59,403 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:59,404 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 47636dde92dbddaf4a0d879337d26302, disabling compactions & flushes 2023-02-13 12:15:59,404 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:15:59,404 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:15:59,404 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. after waiting 0 ms 2023-02-13 12:15:59,404 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:15:59,404 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:15:59,404 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 47636dde92dbddaf4a0d879337d26302: Waiting for close lock at 1676290559404Disabling compacts and flushes for region at 1676290559404Disabling writes for close at 1676290559404Writing region close event to WAL at 1676290559404Closed at 1676290559404 2023-02-13 12:15:59,406 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:15:59,407 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676290559407"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290559407"}]},"ts":"1676290559407"} 2023-02-13 12:15:59,409 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:15:59,409 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:15:59,410 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290559409"}]},"ts":"1676290559409"} 2023-02-13 12:15:59,411 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-13 12:15:59,468 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:15:59,469 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:15:59,469 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:15:59,469 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:15:59,469 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:15:59,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47636dde92dbddaf4a0d879337d26302, ASSIGN}] 2023-02-13 12:15:59,471 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47636dde92dbddaf4a0d879337d26302, ASSIGN 2023-02-13 12:15:59,471 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47636dde92dbddaf4a0d879337d26302, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,45345,1676290533799; forceNewPlan=false, retain=false 2023-02-13 12:15:59,515 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:15:59,622 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:15:59,623 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=47636dde92dbddaf4a0d879337d26302, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:59,623 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676290559623"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290559623"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290559623"}]},"ts":"1676290559623"} 2023-02-13 12:15:59,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 47636dde92dbddaf4a0d879337d26302, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:15:59,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:15:59,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 47636dde92dbddaf4a0d879337d26302, NAME => 'testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:15:59,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:15:59,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 47636dde92dbddaf4a0d879337d26302 2023-02-13 12:15:59,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:15:59,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 47636dde92dbddaf4a0d879337d26302 2023-02-13 12:15:59,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 47636dde92dbddaf4a0d879337d26302 2023-02-13 12:15:59,785 INFO [StoreOpener-47636dde92dbddaf4a0d879337d26302-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 47636dde92dbddaf4a0d879337d26302 2023-02-13 12:15:59,786 DEBUG [StoreOpener-47636dde92dbddaf4a0d879337d26302-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/cf1 2023-02-13 12:15:59,786 DEBUG [StoreOpener-47636dde92dbddaf4a0d879337d26302-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/cf1 2023-02-13 12:15:59,786 INFO [StoreOpener-47636dde92dbddaf4a0d879337d26302-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47636dde92dbddaf4a0d879337d26302 columnFamilyName cf1 2023-02-13 12:15:59,787 INFO [StoreOpener-47636dde92dbddaf4a0d879337d26302-1] regionserver.HStore(310): Store=47636dde92dbddaf4a0d879337d26302/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:59,787 INFO [StoreOpener-47636dde92dbddaf4a0d879337d26302-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 47636dde92dbddaf4a0d879337d26302 2023-02-13 12:15:59,789 DEBUG [StoreOpener-47636dde92dbddaf4a0d879337d26302-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/cf2 2023-02-13 12:15:59,789 DEBUG [StoreOpener-47636dde92dbddaf4a0d879337d26302-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/cf2 2023-02-13 12:15:59,789 INFO [StoreOpener-47636dde92dbddaf4a0d879337d26302-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47636dde92dbddaf4a0d879337d26302 columnFamilyName cf2 2023-02-13 12:15:59,790 INFO [StoreOpener-47636dde92dbddaf4a0d879337d26302-1] regionserver.HStore(310): Store=47636dde92dbddaf4a0d879337d26302/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:15:59,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302 2023-02-13 12:15:59,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302 2023-02-13 12:15:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:15:59,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 47636dde92dbddaf4a0d879337d26302 2023-02-13 12:15:59,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:15:59,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 47636dde92dbddaf4a0d879337d26302; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70959389, jitterRate=0.05737729370594025}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:15:59,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 47636dde92dbddaf4a0d879337d26302: Running coprocessor pre-open hook at 1676290559784Writing region info on filesystem at 1676290559784Initializing all the Stores at 1676290559785 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290559785Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290559785Cleaning up temporary data from old regions at 1676290559791 (+6 ms)Cleaning up detritus from prior splits at 1676290559792 (+1 ms)Running coprocessor post-open hooks at 1676290559795 (+3 ms)Region opened successfully at 1676290559795 2023-02-13 12:15:59,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302., pid=55, masterSystemTime=1676290559776 2023-02-13 12:15:59,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:15:59,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:15:59,798 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=47636dde92dbddaf4a0d879337d26302, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:59,798 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676290559798"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290559798"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290559798"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290559798"}]},"ts":"1676290559798"} 2023-02-13 12:15:59,801 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-13 12:15:59,801 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 47636dde92dbddaf4a0d879337d26302, server=jenkins-hbase16.apache.org,45345,1676290533799 in 174 msec 2023-02-13 12:15:59,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-13 12:15:59,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47636dde92dbddaf4a0d879337d26302, ASSIGN in 332 msec 2023-02-13 12:15:59,803 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:15:59,804 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290559803"}]},"ts":"1676290559803"} 2023-02-13 12:15:59,805 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-13 12:15:59,843 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:15:59,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 464 msec 2023-02-13 12:15:59,881 DEBUG [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:59,882 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:15:59,883 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:15:59,883 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:15:59,883 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:15:59,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47636dde92dbddaf4a0d879337d26302, UNASSIGN}] 2023-02-13 12:15:59,886 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47636dde92dbddaf4a0d879337d26302, UNASSIGN 2023-02-13 12:15:59,886 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=47636dde92dbddaf4a0d879337d26302, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:15:59,886 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676290559886"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290559886"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290559886"}]},"ts":"1676290559886"} 2023-02-13 12:15:59,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 47636dde92dbddaf4a0d879337d26302, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:16:00,040 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 47636dde92dbddaf4a0d879337d26302 2023-02-13 12:16:00,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 47636dde92dbddaf4a0d879337d26302, disabling compactions & flushes 2023-02-13 12:16:00,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:16:00,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:16:00,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. after waiting 0 ms 2023-02-13 12:16:00,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:16:00,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 12:16:00,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:16:00,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302. 2023-02-13 12:16:00,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 47636dde92dbddaf4a0d879337d26302: Waiting for close lock at 1676290560041Running coprocessor pre-close hooks at 1676290560041Disabling compacts and flushes for region at 1676290560041Disabling writes for close at 1676290560042 (+1 ms)Writing region close event to WAL at 1676290560042Running coprocessor post-close hooks at 1676290560085 (+43 ms)Closed at 1676290560085 2023-02-13 12:16:00,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 47636dde92dbddaf4a0d879337d26302 2023-02-13 12:16:00,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=47636dde92dbddaf4a0d879337d26302, regionState=CLOSED 2023-02-13 12:16:00,089 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676290560088"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290560088"}]},"ts":"1676290560088"} 2023-02-13 12:16:00,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-13 12:16:00,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 47636dde92dbddaf4a0d879337d26302, server=jenkins-hbase16.apache.org,45345,1676290533799 in 203 msec 2023-02-13 12:16:00,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-13 12:16:00,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47636dde92dbddaf4a0d879337d26302, UNASSIGN in 209 msec 2023-02-13 12:16:00,131 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 12:16:00,132 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 12:16:00,134 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676290560134"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676290560134"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676290560134"}]},"ts":"1676290560134"} 2023-02-13 12:16:00,134 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290560134"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290560134"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290560134"}]},"ts":"1676290560134"} 2023-02-13 12:16:00,134 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290560134"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290560134"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290560134"}]},"ts":"1676290560134"} 2023-02-13 12:16:00,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=470a6381d0d0d2d0d7d5770c5e13dc82, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b46ec0870b7b2f348fd0578c2ec103c7, ASSIGN}] 2023-02-13 12:16:00,137 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b46ec0870b7b2f348fd0578c2ec103c7, ASSIGN 2023-02-13 12:16:00,138 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b46ec0870b7b2f348fd0578c2ec103c7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,45345,1676290533799; forceNewPlan=false, retain=false 2023-02-13 12:16:00,138 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=470a6381d0d0d2d0d7d5770c5e13dc82, ASSIGN 2023-02-13 12:16:00,138 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=470a6381d0d0d2d0d7d5770c5e13dc82, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,45345,1676290533799; forceNewPlan=false, retain=false 2023-02-13 12:16:00,288 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 12:16:00,291 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=470a6381d0d0d2d0d7d5770c5e13dc82, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:00,292 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290560291"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290560291"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290560291"}]},"ts":"1676290560291"} 2023-02-13 12:16:00,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 470a6381d0d0d2d0d7d5770c5e13dc82, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:16:00,300 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=b46ec0870b7b2f348fd0578c2ec103c7, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:00,300 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290560300"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290560300"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290560300"}]},"ts":"1676290560300"} 2023-02-13 12:16:00,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure b46ec0870b7b2f348fd0578c2ec103c7, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:16:00,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82. 2023-02-13 12:16:00,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 470a6381d0d0d2d0d7d5770c5e13dc82, NAME => 'testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 12:16:00,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:16:00,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:16:00,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,454 INFO [StoreOpener-470a6381d0d0d2d0d7d5770c5e13dc82-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,456 DEBUG [StoreOpener-470a6381d0d0d2d0d7d5770c5e13dc82-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/cf1 2023-02-13 12:16:00,456 DEBUG [StoreOpener-470a6381d0d0d2d0d7d5770c5e13dc82-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/cf1 2023-02-13 12:16:00,456 INFO [StoreOpener-470a6381d0d0d2d0d7d5770c5e13dc82-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 470a6381d0d0d2d0d7d5770c5e13dc82 columnFamilyName cf1 2023-02-13 12:16:00,457 INFO [StoreOpener-470a6381d0d0d2d0d7d5770c5e13dc82-1] regionserver.HStore(310): Store=470a6381d0d0d2d0d7d5770c5e13dc82/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:00,457 INFO [StoreOpener-470a6381d0d0d2d0d7d5770c5e13dc82-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,459 DEBUG [StoreOpener-470a6381d0d0d2d0d7d5770c5e13dc82-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/cf2 2023-02-13 12:16:00,459 DEBUG [StoreOpener-470a6381d0d0d2d0d7d5770c5e13dc82-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/cf2 2023-02-13 12:16:00,459 INFO [StoreOpener-470a6381d0d0d2d0d7d5770c5e13dc82-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 470a6381d0d0d2d0d7d5770c5e13dc82 columnFamilyName cf2 2023-02-13 12:16:00,460 INFO [StoreOpener-470a6381d0d0d2d0d7d5770c5e13dc82-1] regionserver.HStore(310): Store=470a6381d0d0d2d0d7d5770c5e13dc82/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:00,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:16:00,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 470a6381d0d0d2d0d7d5770c5e13dc82; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70301691, jitterRate=0.04757682979106903}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:16:00,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 470a6381d0d0d2d0d7d5770c5e13dc82: Running coprocessor pre-open hook at 1676290560452Writing region info on filesystem at 1676290560452Initializing all the Stores at 1676290560453 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290560454 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290560454Cleaning up temporary data from old regions at 1676290560462 (+8 ms)Cleaning up detritus from prior splits at 1676290560463 (+1 ms)Running coprocessor post-open hooks at 1676290560466 (+3 ms)Region opened successfully at 1676290560466 2023-02-13 12:16:00,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82., pid=61, masterSystemTime=1676290560448 2023-02-13 12:16:00,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82. 2023-02-13 12:16:00,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82. 2023-02-13 12:16:00,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7. 2023-02-13 12:16:00,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b46ec0870b7b2f348fd0578c2ec103c7, NAME => 'testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 12:16:00,468 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=470a6381d0d0d2d0d7d5770c5e13dc82, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:00,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:16:00,468 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290560468"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290560468"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290560468"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290560468"}]},"ts":"1676290560468"} 2023-02-13 12:16:00,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:16:00,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,471 INFO [StoreOpener-b46ec0870b7b2f348fd0578c2ec103c7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,473 DEBUG [StoreOpener-b46ec0870b7b2f348fd0578c2ec103c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/cf1 2023-02-13 12:16:00,473 DEBUG [StoreOpener-b46ec0870b7b2f348fd0578c2ec103c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/cf1 2023-02-13 12:16:00,474 INFO [StoreOpener-b46ec0870b7b2f348fd0578c2ec103c7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b46ec0870b7b2f348fd0578c2ec103c7 columnFamilyName cf1 2023-02-13 12:16:00,474 INFO [StoreOpener-b46ec0870b7b2f348fd0578c2ec103c7-1] regionserver.HStore(310): Store=b46ec0870b7b2f348fd0578c2ec103c7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:00,475 INFO [StoreOpener-b46ec0870b7b2f348fd0578c2ec103c7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-13 12:16:00,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 470a6381d0d0d2d0d7d5770c5e13dc82, server=jenkins-hbase16.apache.org,45345,1676290533799 in 175 msec 2023-02-13 12:16:00,476 DEBUG [StoreOpener-b46ec0870b7b2f348fd0578c2ec103c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/cf2 2023-02-13 12:16:00,476 DEBUG [StoreOpener-b46ec0870b7b2f348fd0578c2ec103c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/cf2 2023-02-13 12:16:00,477 INFO [StoreOpener-b46ec0870b7b2f348fd0578c2ec103c7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b46ec0870b7b2f348fd0578c2ec103c7 columnFamilyName cf2 2023-02-13 12:16:00,477 INFO [StoreOpener-b46ec0870b7b2f348fd0578c2ec103c7-1] regionserver.HStore(310): Store=b46ec0870b7b2f348fd0578c2ec103c7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:00,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=470a6381d0d0d2d0d7d5770c5e13dc82, ASSIGN in 339 msec 2023-02-13 12:16:00,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:16:00,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened b46ec0870b7b2f348fd0578c2ec103c7; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61124594, jitterRate=-0.08917257189750671}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:16:00,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for b46ec0870b7b2f348fd0578c2ec103c7: Running coprocessor pre-open hook at 1676290560469Writing region info on filesystem at 1676290560469Initializing all the Stores at 1676290560470 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290560470Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290560471 (+1 ms)Cleaning up temporary data from old regions at 1676290560480 (+9 ms)Cleaning up detritus from prior splits at 1676290560481 (+1 ms)Running coprocessor post-open hooks at 1676290560483 (+2 ms)Region opened successfully at 1676290560483 2023-02-13 12:16:00,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7., pid=62, masterSystemTime=1676290560448 2023-02-13 12:16:00,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7. 2023-02-13 12:16:00,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7. 2023-02-13 12:16:00,485 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=b46ec0870b7b2f348fd0578c2ec103c7, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:00,486 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290560485"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290560485"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290560485"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290560485"}]},"ts":"1676290560485"} 2023-02-13 12:16:00,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-13 12:16:00,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure b46ec0870b7b2f348fd0578c2ec103c7, server=jenkins-hbase16.apache.org,45345,1676290533799 in 185 msec 2023-02-13 12:16:00,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-02-13 12:16:00,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b46ec0870b7b2f348fd0578c2ec103c7, ASSIGN in 352 msec 2023-02-13 12:16:00,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 in 609 msec 2023-02-13 12:16:00,638 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 12:16:00,642 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:16:00,643 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-13 12:16:00,643 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitTableRegionNoStoreFile 2023-02-13 12:16:00,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:16:00,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 12:16:00,647 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290560647"}]},"ts":"1676290560647"} 2023-02-13 12:16:00,648 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-13 12:16:00,658 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-13 12:16:00,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=470a6381d0d0d2d0d7d5770c5e13dc82, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b46ec0870b7b2f348fd0578c2ec103c7, UNASSIGN}] 2023-02-13 12:16:00,660 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b46ec0870b7b2f348fd0578c2ec103c7, UNASSIGN 2023-02-13 12:16:00,661 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=b46ec0870b7b2f348fd0578c2ec103c7, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:00,661 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290560661"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290560661"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290560661"}]},"ts":"1676290560661"} 2023-02-13 12:16:00,663 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure b46ec0870b7b2f348fd0578c2ec103c7, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:16:00,665 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=470a6381d0d0d2d0d7d5770c5e13dc82, UNASSIGN 2023-02-13 12:16:00,666 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=470a6381d0d0d2d0d7d5770c5e13dc82, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:00,666 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290560666"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290560666"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290560666"}]},"ts":"1676290560666"} 2023-02-13 12:16:00,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 470a6381d0d0d2d0d7d5770c5e13dc82, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:16:00,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 12:16:00,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 470a6381d0d0d2d0d7d5770c5e13dc82, disabling compactions & flushes 2023-02-13 12:16:00,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82. 2023-02-13 12:16:00,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82. 2023-02-13 12:16:00,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82. after waiting 0 ms 2023-02-13 12:16:00,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82. 2023-02-13 12:16:00,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 12:16:00,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:16:00,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82. 2023-02-13 12:16:00,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 470a6381d0d0d2d0d7d5770c5e13dc82: Waiting for close lock at 1676290560815Running coprocessor pre-close hooks at 1676290560815Disabling compacts and flushes for region at 1676290560815Disabling writes for close at 1676290560816 (+1 ms)Writing region close event to WAL at 1676290560817 (+1 ms)Running coprocessor post-close hooks at 1676290560823 (+6 ms)Closed at 1676290560823 2023-02-13 12:16:00,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing b46ec0870b7b2f348fd0578c2ec103c7, disabling compactions & flushes 2023-02-13 12:16:00,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7. 2023-02-13 12:16:00,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7. 2023-02-13 12:16:00,826 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=470a6381d0d0d2d0d7d5770c5e13dc82, regionState=CLOSED 2023-02-13 12:16:00,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7. after waiting 0 ms 2023-02-13 12:16:00,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7. 2023-02-13 12:16:00,827 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290560826"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290560826"}]},"ts":"1676290560826"} 2023-02-13 12:16:00,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-13 12:16:00,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 470a6381d0d0d2d0d7d5770c5e13dc82, server=jenkins-hbase16.apache.org,45345,1676290533799 in 160 msec 2023-02-13 12:16:00,831 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 12:16:00,831 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:16:00,831 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7. 2023-02-13 12:16:00,831 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for b46ec0870b7b2f348fd0578c2ec103c7: Waiting for close lock at 1676290560825Running coprocessor pre-close hooks at 1676290560825Disabling compacts and flushes for region at 1676290560825Disabling writes for close at 1676290560827 (+2 ms)Writing region close event to WAL at 1676290560827Running coprocessor post-close hooks at 1676290560831 (+4 ms)Closed at 1676290560831 2023-02-13 12:16:00,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=470a6381d0d0d2d0d7d5770c5e13dc82, UNASSIGN in 171 msec 2023-02-13 12:16:00,833 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,833 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=b46ec0870b7b2f348fd0578c2ec103c7, regionState=CLOSED 2023-02-13 12:16:00,833 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290560833"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290560833"}]},"ts":"1676290560833"} 2023-02-13 12:16:00,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-13 12:16:00,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure b46ec0870b7b2f348fd0578c2ec103c7, server=jenkins-hbase16.apache.org,45345,1676290533799 in 171 msec 2023-02-13 12:16:00,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-13 12:16:00,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b46ec0870b7b2f348fd0578c2ec103c7, UNASSIGN in 177 msec 2023-02-13 12:16:00,838 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290560837"}]},"ts":"1676290560837"} 2023-02-13 12:16:00,839 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-13 12:16:00,847 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-13 12:16:00,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 204 msec 2023-02-13 12:16:00,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 12:16:00,950 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-13 12:16:00,951 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitTableRegionNoStoreFile 2023-02-13 12:16:00,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:16:00,954 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:16:00,955 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:16:00,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 12:16:00,961 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302 2023-02-13 12:16:00,962 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,962 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,964 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/recovered.edits] 2023-02-13 12:16:00,964 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/recovered.edits] 2023-02-13 12:16:00,965 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/recovered.edits] 2023-02-13 12:16:00,976 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/recovered.edits/7.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7/recovered.edits/7.seqid 2023-02-13 12:16:00,976 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/recovered.edits/7.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82/recovered.edits/7.seqid 2023-02-13 12:16:00,976 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/recovered.edits/4.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302/recovered.edits/4.seqid 2023-02-13 12:16:00,977 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:16:00,977 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/470a6381d0d0d2d0d7d5770c5e13dc82 2023-02-13 12:16:00,977 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionNoStoreFile/47636dde92dbddaf4a0d879337d26302 2023-02-13 12:16:00,977 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 12:16:00,979 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:16:00,981 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-13 12:16:00,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-13 12:16:00,985 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:16:00,985 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-13 12:16:00,985 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290560985"}]},"ts":"9223372036854775807"} 2023-02-13 12:16:00,985 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290560985"}]},"ts":"9223372036854775807"} 2023-02-13 12:16:00,985 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290560985"}]},"ts":"9223372036854775807"} 2023-02-13 12:16:00,987 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 12:16:00,987 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 47636dde92dbddaf4a0d879337d26302, NAME => 'testSplitTableRegionNoStoreFile,,1676290559379.47636dde92dbddaf4a0d879337d26302.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 470a6381d0d0d2d0d7d5770c5e13dc82, NAME => 'testSplitTableRegionNoStoreFile,,1676290559881.470a6381d0d0d2d0d7d5770c5e13dc82.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => b46ec0870b7b2f348fd0578c2ec103c7, NAME => 'testSplitTableRegionNoStoreFile,41,1676290559881.b46ec0870b7b2f348fd0578c2ec103c7.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 12:16:00,987 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-13 12:16:00,987 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676290560987"}]},"ts":"9223372036854775807"} 2023-02-13 12:16:00,989 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-13 12:16:00,998 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:16:00,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 47 msec 2023-02-13 12:16:01,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 12:16:01,057 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-13 12:16:01,070 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=408 (was 402) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_363941975_15 at /127.0.0.1:46504 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_363941975_15 at /127.0.0.1:42818 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=849 (was 847) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=335 (was 321) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=5370 (was 5431) 2023-02-13 12:16:01,084 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=408, OpenFileDescriptor=849, MaxFileDescriptor=60000, SystemLoadAverage=335, ProcessCount=167, AvailableMemoryMB=5368 2023-02-13 12:16:01,084 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:16:01,092 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 12:16:01,092 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 12:16:01,093 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,095 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:16:01,099 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:16:01,101 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:01,101 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66 empty. 2023-02-13 12:16:01,102 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:01,102 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 12:16:01,123 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 12:16:01,125 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => b4de9409ed4174da452e463dd5faeb66, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:16:01,151 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:16:01,151 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing b4de9409ed4174da452e463dd5faeb66, disabling compactions & flushes 2023-02-13 12:16:01,152 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:01,152 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:01,152 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. after waiting 0 ms 2023-02-13 12:16:01,152 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:01,152 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:01,152 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for b4de9409ed4174da452e463dd5faeb66: Waiting for close lock at 1676290561151Disabling compacts and flushes for region at 1676290561151Disabling writes for close at 1676290561152 (+1 ms)Writing region close event to WAL at 1676290561152Closed at 1676290561152 2023-02-13 12:16:01,158 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:16:01,159 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676290561159"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290561159"}]},"ts":"1676290561159"} 2023-02-13 12:16:01,161 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:16:01,162 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:16:01,162 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290561162"}]},"ts":"1676290561162"} 2023-02-13 12:16:01,165 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-13 12:16:01,197 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:16:01,199 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:16:01,199 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:16:01,199 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:16:01,199 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:16:01,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b4de9409ed4174da452e463dd5faeb66, ASSIGN}] 2023-02-13 12:16:01,201 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b4de9409ed4174da452e463dd5faeb66, ASSIGN 2023-02-13 12:16:01,202 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b4de9409ed4174da452e463dd5faeb66, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:16:01,352 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:16:01,353 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=b4de9409ed4174da452e463dd5faeb66, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:01,354 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676290561353"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290561353"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290561353"}]},"ts":"1676290561353"} 2023-02-13 12:16:01,355 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure b4de9409ed4174da452e463dd5faeb66, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:16:01,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:01,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b4de9409ed4174da452e463dd5faeb66, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:16:01,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:16:01,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:01,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:16:01,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:01,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:01,536 INFO [StoreOpener-b4de9409ed4174da452e463dd5faeb66-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:01,540 DEBUG [StoreOpener-b4de9409ed4174da452e463dd5faeb66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1 2023-02-13 12:16:01,540 DEBUG [StoreOpener-b4de9409ed4174da452e463dd5faeb66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1 2023-02-13 12:16:01,541 INFO [StoreOpener-b4de9409ed4174da452e463dd5faeb66-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b4de9409ed4174da452e463dd5faeb66 columnFamilyName cf1 2023-02-13 12:16:01,542 INFO [StoreOpener-b4de9409ed4174da452e463dd5faeb66-1] regionserver.HStore(310): Store=b4de9409ed4174da452e463dd5faeb66/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:01,543 INFO [StoreOpener-b4de9409ed4174da452e463dd5faeb66-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:01,545 DEBUG [StoreOpener-b4de9409ed4174da452e463dd5faeb66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2 2023-02-13 12:16:01,545 DEBUG [StoreOpener-b4de9409ed4174da452e463dd5faeb66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2 2023-02-13 12:16:01,546 INFO [StoreOpener-b4de9409ed4174da452e463dd5faeb66-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b4de9409ed4174da452e463dd5faeb66 columnFamilyName cf2 2023-02-13 12:16:01,548 INFO [StoreOpener-b4de9409ed4174da452e463dd5faeb66-1] regionserver.HStore(310): Store=b4de9409ed4174da452e463dd5faeb66/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:01,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:01,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:01,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:16:01,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:01,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:16:01,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened b4de9409ed4174da452e463dd5faeb66; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61635686, jitterRate=-0.08155670762062073}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:16:01,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for b4de9409ed4174da452e463dd5faeb66: Running coprocessor pre-open hook at 1676290561521Writing region info on filesystem at 1676290561521Initializing all the Stores at 1676290561531 (+10 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290561531Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290561536 (+5 ms)Cleaning up temporary data from old regions at 1676290561550 (+14 ms)Cleaning up detritus from prior splits at 1676290561550Running coprocessor post-open hooks at 1676290561565 (+15 ms)Region opened successfully at 1676290561565 2023-02-13 12:16:01,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66., pid=71, masterSystemTime=1676290561507 2023-02-13 12:16:01,568 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:01,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:01,569 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=b4de9409ed4174da452e463dd5faeb66, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:01,569 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676290561569"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290561569"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290561569"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290561569"}]},"ts":"1676290561569"} 2023-02-13 12:16:01,572 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-13 12:16:01,572 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure b4de9409ed4174da452e463dd5faeb66, server=jenkins-hbase16.apache.org,33055,1676290533708 in 216 msec 2023-02-13 12:16:01,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-13 12:16:01,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b4de9409ed4174da452e463dd5faeb66, ASSIGN in 374 msec 2023-02-13 12:16:01,575 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:16:01,575 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290561575"}]},"ts":"1676290561575"} 2023-02-13 12:16:01,576 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-13 12:16:01,584 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:16:01,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 491 msec 2023-02-13 12:16:01,602 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:16:01,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,603 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:01,603 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:01,603 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 12:16:01,603 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 12:16:01,604 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,604 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,614 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:01,614 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:01,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:01,614 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:01,614 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:01,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:01,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,614 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:01,615 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,615 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,616 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:01,616 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:01,616 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:01,616 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:01,620 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:16:01,620 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:16:01,620 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:01,620 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. started... 2023-02-13 12:16:01,621 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b4de9409ed4174da452e463dd5faeb66 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 12:16:01,625 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,633 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,633 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,633 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:01,634 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:01,634 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:01,636 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/634f91df36ca46daa9608a034818fb6e 2023-02-13 12:16:02,110 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/ad90236225cd42c48c874fe838772640 2023-02-13 12:16:02,115 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/634f91df36ca46daa9608a034818fb6e as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e 2023-02-13 12:16:02,119 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:16:02,120 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/ad90236225cd42c48c874fe838772640 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640 2023-02-13 12:16:02,124 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:16:02,125 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 504ms, sequenceid=6, compaction requested=false 2023-02-13 12:16:02,125 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:02,125 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b4de9409ed4174da452e463dd5faeb66: 2023-02-13 12:16:02,125 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:02,125 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:16:02,125 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:16:02,125 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,126 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:02,126 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:02,133 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,133 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:02,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:02,134 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,134 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:02,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:02,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:02,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:02,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 12:16:02,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64b814b[Count = 0] remaining members to acquire global barrier 2023-02-13 12:16:02,135 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 12:16:02,135 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,142 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,142 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,142 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,142 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,142 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,142 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,142 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,142 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:16:02,142 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:16:02,142 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,142 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:16:02,142 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,142 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,142 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:16:02,150 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,150 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,150 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:02,158 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,158 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,158 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:02,158 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,158 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:02,159 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,159 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:16:02,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:16:02,162 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 12:16:02,162 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:16:02,162 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:16:02,162 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,162 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:16:02,172 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,172 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,172 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,172 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,172 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:02,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,172 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,172 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:02,173 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,173 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,174 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:02,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,175 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,175 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,175 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:02,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,211 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:16:02,211 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,212 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,212 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:16:02,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,212 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,212 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,312 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:16:02,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:16:02,331 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:16:02,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:16:02,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,333 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:02,333 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,333 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 12:16:02,333 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 12:16:02,334 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,334 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,342 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,342 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,342 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:16:02,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:02,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,343 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:02,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:02,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:16:02,344 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:16:02,344 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:02,345 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. started... 2023-02-13 12:16:02,345 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b4de9409ed4174da452e463dd5faeb66 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:16:02,350 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,358 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,358 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,358 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:02,359 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,359 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/3253afbef5f8450fafe3da95ddc9a73a 2023-02-13 12:16:02,359 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:02,370 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/9643bc3d56f3426fbfed9aad68002dc3 2023-02-13 12:16:02,374 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/3253afbef5f8450fafe3da95ddc9a73a as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a 2023-02-13 12:16:02,379 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:16:02,380 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/9643bc3d56f3426fbfed9aad68002dc3 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3 2023-02-13 12:16:02,387 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:16:02,388 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 43ms, sequenceid=19, compaction requested=false 2023-02-13 12:16:02,389 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b4de9409ed4174da452e463dd5faeb66: 2023-02-13 12:16:02,389 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:02,389 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:16:02,389 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:16:02,390 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,390 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:02,390 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:02,400 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,400 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:02,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:02,401 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,401 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:02,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:02,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:02,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:02,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 12:16:02,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@201936c7[Count = 0] remaining members to acquire global barrier 2023-02-13 12:16:02,402 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 12:16:02,402 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,414 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,414 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,414 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,414 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,414 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:16:02,414 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,414 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,414 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:16:02,414 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,415 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,415 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:16:02,415 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,415 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,415 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:16:02,431 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,431 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,431 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:02,439 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,439 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,439 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,439 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:02,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:02,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:02,456 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:02,456 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,456 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:02,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:02,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:16:02,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:16:02,460 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 12:16:02,460 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:16:02,460 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:16:02,460 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,460 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:16:02,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:02,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,468 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,469 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:02,469 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:02,471 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,471 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,471 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,489 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,489 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,489 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,489 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,489 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,489 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:02,490 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:16:02,489 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,489 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,489 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,489 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,490 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:16:02,490 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,490 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:16:02,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,491 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,491 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,491 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,491 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,491 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,491 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,491 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,492 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,591 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:16:02,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:16:02,609 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:16:02,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:16:02,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,612 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:02,612 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,612 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 12:16:02,612 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 12:16:02,612 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,612 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,622 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,622 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,622 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,622 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,623 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,623 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:16:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,624 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:02,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:02,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,625 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:02,625 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,625 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:02,625 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:02,625 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:02,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:02,625 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:02,625 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,625 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:02,625 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:02,626 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,626 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:02,626 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:02,626 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:02,626 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:16:02,626 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:16:02,626 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:02,626 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. started... 2023-02-13 12:16:02,626 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b4de9409ed4174da452e463dd5faeb66 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:16:02,640 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/6b95bc15dab94af193614bf89dc1bdc0 2023-02-13 12:16:02,653 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,667 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,667 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,667 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:02,667 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,667 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:02,669 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/c89e30139b444d7f8b0317055459dd2c 2023-02-13 12:16:02,674 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/6b95bc15dab94af193614bf89dc1bdc0 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0 2023-02-13 12:16:02,679 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:16:02,680 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/c89e30139b444d7f8b0317055459dd2c as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c 2023-02-13 12:16:02,686 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:16:02,687 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 61ms, sequenceid=32, compaction requested=true 2023-02-13 12:16:02,687 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b4de9409ed4174da452e463dd5faeb66: 2023-02-13 12:16:02,687 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:02,687 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:16:02,687 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:16:02,687 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,687 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:02,687 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:02,697 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,697 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:02,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:02,698 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,698 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:02,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 12:16:02,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c243734[Count = 0] remaining members to acquire global barrier 2023-02-13 12:16:02,699 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 12:16:02,699 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,714 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,714 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,714 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,714 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,715 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,715 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:16:02,714 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,715 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:16:02,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,715 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,715 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,715 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,715 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,715 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:16:02,715 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:16:02,731 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,731 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,731 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:02,731 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:02,739 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,739 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,739 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:02,747 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:02,747 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,747 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:02,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:02,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:02,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:16:02,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:16:02,753 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 12:16:02,753 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:16:02,753 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:16:02,753 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,753 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:16:02,764 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,764 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:02,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:02,764 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,764 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,765 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,764 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,765 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:02,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,765 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,766 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,767 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:02,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:02,771 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,771 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,772 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,783 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,783 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,783 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,783 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,783 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,784 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:02,784 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:16:02,783 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,783 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,783 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,784 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,784 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:16:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,784 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,784 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,784 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:16:02,784 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,785 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:02,785 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,785 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,785 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,785 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,785 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,885 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:16:02,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:16:02,903 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:16:02,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:16:02,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,905 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:02,905 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,905 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 12:16:02,905 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 12:16:02,905 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,905 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,914 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,914 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,914 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,914 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:02,914 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,914 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:16:02,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:02,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:02,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,915 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,916 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,916 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:02,916 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:02,916 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,916 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:02,916 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,916 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:02,916 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:02,917 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:16:02,917 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:16:02,917 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:02,917 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. started... 2023-02-13 12:16:02,917 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b4de9409ed4174da452e463dd5faeb66 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:16:02,925 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,928 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/dffdf58afd9c40c49a26273b78a31699 2023-02-13 12:16:02,933 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,933 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,933 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:02,934 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,934 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:02,944 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/b096fd687bae4b0d8ac5f950d1a105ab 2023-02-13 12:16:02,948 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/dffdf58afd9c40c49a26273b78a31699 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/dffdf58afd9c40c49a26273b78a31699 2023-02-13 12:16:02,952 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/dffdf58afd9c40c49a26273b78a31699, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:16:02,953 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/b096fd687bae4b0d8ac5f950d1a105ab as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/b096fd687bae4b0d8ac5f950d1a105ab 2023-02-13 12:16:02,957 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/b096fd687bae4b0d8ac5f950d1a105ab, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:16:02,958 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 41ms, sequenceid=45, compaction requested=true 2023-02-13 12:16:02,958 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b4de9409ed4174da452e463dd5faeb66: 2023-02-13 12:16:02,958 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:02,958 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:16:02,958 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:16:02,958 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,958 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:02,958 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:02,967 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,967 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:02,967 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,967 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:02,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:02,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 12:16:02,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1dea2bee[Count = 0] remaining members to acquire global barrier 2023-02-13 12:16:02,969 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 12:16:02,969 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,975 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,975 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,975 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,975 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,975 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,975 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:16:02,975 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,975 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,975 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:02,975 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:16:02,975 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:16:02,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:02,976 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:02,976 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:02,976 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:16:02,983 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,983 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,983 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:02,992 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:02,992 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:02,992 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,022 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:03,022 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,022 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,022 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:16:03,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:16:03,030 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 12:16:03,030 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:16:03,030 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:16:03,030 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,030 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:16:03,042 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,042 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,042 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,042 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,042 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,042 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,042 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,043 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,044 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,045 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,049 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,049 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,050 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,059 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:03,059 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:16:03,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,059 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,059 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:16:03,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,059 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:16:03,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,059 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,060 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,060 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,060 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,060 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,160 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:16:03,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:16:03,185 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:16:03,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:16:03,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,187 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:03,187 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,187 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 12:16:03,187 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 12:16:03,188 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,188 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,197 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,197 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,197 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,197 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,198 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,198 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:16:03,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,199 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,199 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,199 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,199 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,200 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,200 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,200 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,200 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,200 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,200 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,200 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,200 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,200 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,200 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,200 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,201 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,201 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,201 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,201 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,201 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,201 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:16:03,202 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:03,202 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,202 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,202 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. started... 2023-02-13 12:16:03,202 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:16:03,202 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,202 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,202 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b4de9409ed4174da452e463dd5faeb66 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:16:03,202 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,208 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,209 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,209 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,214 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/ca8b931880134e96ab7a5adfead3b64b 2023-02-13 12:16:03,217 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,217 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,217 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,226 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/d76b5448f9034a0a83588157300f5dd7 2023-02-13 12:16:03,233 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/ca8b931880134e96ab7a5adfead3b64b as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/ca8b931880134e96ab7a5adfead3b64b 2023-02-13 12:16:03,237 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/ca8b931880134e96ab7a5adfead3b64b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:16:03,238 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/d76b5448f9034a0a83588157300f5dd7 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d76b5448f9034a0a83588157300f5dd7 2023-02-13 12:16:03,243 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d76b5448f9034a0a83588157300f5dd7, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:16:03,244 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 42ms, sequenceid=58, compaction requested=true 2023-02-13 12:16:03,244 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b4de9409ed4174da452e463dd5faeb66: 2023-02-13 12:16:03,244 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:03,244 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:16:03,244 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:16:03,244 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,244 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,244 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,250 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,250 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,250 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,250 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 12:16:03,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47f1b0c1[Count = 0] remaining members to acquire global barrier 2023-02-13 12:16:03,254 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 12:16:03,254 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,264 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,264 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,264 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,264 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,264 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:16:03,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,264 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:03,264 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:03,264 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:03,264 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:03,265 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:16:03,264 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:03,265 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:03,265 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:16:03,265 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:16:03,275 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:03,275 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,275 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,275 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,283 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:03,283 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:03,283 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,283 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,283 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,284 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:16:03,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:16:03,286 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 12:16:03,286 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:16:03,286 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:16:03,286 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,286 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:16:03,300 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,300 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,300 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,300 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,300 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,300 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,300 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,301 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,301 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,301 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,302 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,303 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,303 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,333 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,333 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,333 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,333 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,334 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:03,334 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:16:03,333 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,333 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,333 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,333 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,333 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,334 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:16:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,334 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,334 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:16:03,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,334 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,334 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,334 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,335 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,335 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,335 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,335 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,335 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,434 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:16:03,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:16:03,454 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:16:03,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:16:03,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,456 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:03,456 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,456 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 12:16:03,456 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 12:16:03,457 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,457 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,467 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,468 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,468 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,468 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,469 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:16:03,469 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,469 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. started... 2023-02-13 12:16:03,469 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b4de9409ed4174da452e463dd5faeb66 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:16:03,471 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,475 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,483 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,483 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,483 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,483 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,483 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:16:03,484 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,484 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,484 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/1f5893517ddf4a11807c732097dc7f74 2023-02-13 12:16:03,498 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/6ef545109dcf47e0b9383bce3d2bc0f3 2023-02-13 12:16:03,502 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/1f5893517ddf4a11807c732097dc7f74 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/1f5893517ddf4a11807c732097dc7f74 2023-02-13 12:16:03,506 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/1f5893517ddf4a11807c732097dc7f74, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:16:03,507 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/6ef545109dcf47e0b9383bce3d2bc0f3 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6ef545109dcf47e0b9383bce3d2bc0f3 2023-02-13 12:16:03,511 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6ef545109dcf47e0b9383bce3d2bc0f3, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:16:03,512 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 43ms, sequenceid=71, compaction requested=true 2023-02-13 12:16:03,512 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b4de9409ed4174da452e463dd5faeb66: 2023-02-13 12:16:03,512 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:03,512 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:16:03,512 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:16:03,512 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,512 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,512 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,522 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,522 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,523 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,523 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,525 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 12:16:03,525 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e3719f6[Count = 0] remaining members to acquire global barrier 2023-02-13 12:16:03,525 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 12:16:03,525 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,533 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,533 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,533 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,534 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:03,534 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,534 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:16:03,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,534 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:03,534 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:03,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,534 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:03,534 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:16:03,534 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:16:03,534 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:03,534 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:03,534 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:16:03,542 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:03,542 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,542 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:03,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,542 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,542 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,542 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,542 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,550 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:03,550 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,550 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:16:03,553 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:16:03,553 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 12:16:03,553 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:16:03,553 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:16:03,553 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,553 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:16:03,558 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,558 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,558 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,558 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,558 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,559 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,559 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,560 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,560 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,560 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,562 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,562 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,562 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,572 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,572 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,572 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,572 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,572 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,572 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:03,572 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:16:03,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,572 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,572 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:16:03,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,572 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,572 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:16:03,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,573 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,673 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:16:03,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:16:03,686 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:16:03,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:16:03,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,688 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:03,688 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,688 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 12:16:03,688 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 12:16:03,688 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,688 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,697 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,697 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,697 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,697 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,698 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,698 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,699 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,699 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,699 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,699 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,699 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,699 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,699 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,699 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,699 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:16:03,699 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. started... 2023-02-13 12:16:03,699 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,700 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,700 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b4de9409ed4174da452e463dd5faeb66 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-13 12:16:03,708 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,709 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,709 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,709 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/304056b95f9f43c39c48a011cc38f134 2023-02-13 12:16:03,713 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 304056b95f9f43c39c48a011cc38f134 2023-02-13 12:16:03,717 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,717 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,717 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,722 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/a2c5d0fb72224e17bbca80ffdf8c384e 2023-02-13 12:16:03,726 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2c5d0fb72224e17bbca80ffdf8c384e 2023-02-13 12:16:03,726 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/304056b95f9f43c39c48a011cc38f134 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134 2023-02-13 12:16:03,730 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 304056b95f9f43c39c48a011cc38f134 2023-02-13 12:16:03,730 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 12:16:03,731 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/a2c5d0fb72224e17bbca80ffdf8c384e as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e 2023-02-13 12:16:03,735 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2c5d0fb72224e17bbca80ffdf8c384e 2023-02-13 12:16:03,735 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 12:16:03,735 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 35ms, sequenceid=83, compaction requested=true 2023-02-13 12:16:03,735 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b4de9409ed4174da452e463dd5faeb66: 2023-02-13 12:16:03,736 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:03,736 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:16:03,736 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:16:03,736 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,736 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,736 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,742 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,742 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,742 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,742 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 12:16:03,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3071693[Count = 0] remaining members to acquire global barrier 2023-02-13 12:16:03,744 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 12:16:03,744 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,750 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,750 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,750 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,750 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:03,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,750 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:03,750 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,750 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:16:03,750 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:03,750 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:16:03,750 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:03,750 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:03,750 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:03,751 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:16:03,751 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:16:03,785 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,785 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:03,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,785 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,785 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,797 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:03,797 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,797 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,806 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:03,806 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,806 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:03,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:16:03,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:16:03,809 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 12:16:03,809 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:16:03,809 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:16:03,809 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,809 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:16:03,817 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,817 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,817 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,817 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:03,817 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,817 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:03,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,817 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,818 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,818 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,818 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:03,819 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,820 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,820 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,831 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,831 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,831 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,831 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:03,831 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,831 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,831 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,831 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,831 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:03,831 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:16:03,831 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,831 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:03,831 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,832 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:16:03,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,832 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,832 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:16:03,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:03,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:03,832 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:03,832 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,832 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:03,832 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,832 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,832 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,932 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:16:03,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:16:03,940 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:16:03,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:16:03,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,942 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:03,942 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,942 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 12:16:03,942 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 12:16:03,942 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,942 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,950 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,950 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,950 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,950 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:03,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,950 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,950 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:16:03,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:03,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:03,952 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,952 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,952 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,952 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,952 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,952 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,952 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,952 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:03,952 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,953 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:03,953 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:03,953 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:16:03,953 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:16:03,953 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:03,953 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. started... 2023-02-13 12:16:03,953 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b4de9409ed4174da452e463dd5faeb66 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 12:16:03,958 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,964 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/d41689a539d843338802577ea77e7b02 2023-02-13 12:16:03,967 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,967 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,967 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,967 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:03,967 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:03,969 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41689a539d843338802577ea77e7b02 2023-02-13 12:16:03,980 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/6013eec329b94144aff522e8a6c824aa 2023-02-13 12:16:03,985 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6013eec329b94144aff522e8a6c824aa 2023-02-13 12:16:03,986 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/d41689a539d843338802577ea77e7b02 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/d41689a539d843338802577ea77e7b02 2023-02-13 12:16:03,992 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41689a539d843338802577ea77e7b02 2023-02-13 12:16:03,992 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/d41689a539d843338802577ea77e7b02, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 12:16:03,993 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/6013eec329b94144aff522e8a6c824aa as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6013eec329b94144aff522e8a6c824aa 2023-02-13 12:16:03,996 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6013eec329b94144aff522e8a6c824aa 2023-02-13 12:16:03,997 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6013eec329b94144aff522e8a6c824aa, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 12:16:03,997 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 44ms, sequenceid=91, compaction requested=true 2023-02-13 12:16:03,997 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b4de9409ed4174da452e463dd5faeb66: 2023-02-13 12:16:03,997 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:03,997 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:16:03,997 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:16:03,998 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:03,998 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:03,998 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:04,008 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,008 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:04,009 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:04,009 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:04,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:04,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:04,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 12:16:04,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ffee846[Count = 0] remaining members to acquire global barrier 2023-02-13 12:16:04,012 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 12:16:04,012 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,022 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,022 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,022 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,023 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,023 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:16:04,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,023 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:04,023 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:04,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,023 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:16:04,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,024 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:04,024 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:04,024 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:04,024 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:04,024 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:16:04,024 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:16:04,034 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,034 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:04,034 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:04,034 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:04,042 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:04,042 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:04,042 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:04,042 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:04,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:04,042 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:04,043 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:04,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:04,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:16:04,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:16:04,052 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 12:16:04,052 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:16:04,052 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:16:04,052 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,052 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:16:04,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,058 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,059 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:04,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,059 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,060 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:04,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:04,061 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,061 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,061 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,094 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:04,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:04,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:04,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:04,095 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:04,095 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:16:04,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:04,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:04,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,095 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:04,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:04,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:04,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,095 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:16:04,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,096 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:16:04,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,096 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,097 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,097 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,097 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,097 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,097 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,097 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,197 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:16:04,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:16:04,216 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:16:04,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:16:04,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,218 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:04,218 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:04,218 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 12:16:04,218 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 12:16:04,219 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,219 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,233 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:04,233 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:04,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:04,233 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,233 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:04,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:04,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,234 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,234 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,235 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:04,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:16:04,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,235 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:04,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,235 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,235 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,235 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:04,235 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:04,235 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,235 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:04,235 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:04,235 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:04,235 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 12:16:04,235 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:04,235 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,235 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,235 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:04,235 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 12:16:04,235 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:04,235 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:16:04,236 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:04,236 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:04,236 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:04,236 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 12:16:04,236 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:16:04,236 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:16:04,236 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:04,236 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. started... 2023-02-13 12:16:04,236 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b4de9409ed4174da452e463dd5faeb66 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 12:16:04,242 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,248 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/6ef1a73a8e0644b7a2cc946415ba5ea2 2023-02-13 12:16:04,250 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,250 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,250 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:04,250 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,250 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:04,252 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ef1a73a8e0644b7a2cc946415ba5ea2 2023-02-13 12:16:04,261 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/d3a13caf3b644632b250a0ec762caf04 2023-02-13 12:16:04,265 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3a13caf3b644632b250a0ec762caf04 2023-02-13 12:16:04,265 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/6ef1a73a8e0644b7a2cc946415ba5ea2 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6ef1a73a8e0644b7a2cc946415ba5ea2 2023-02-13 12:16:04,269 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ef1a73a8e0644b7a2cc946415ba5ea2 2023-02-13 12:16:04,269 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6ef1a73a8e0644b7a2cc946415ba5ea2, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 12:16:04,270 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/d3a13caf3b644632b250a0ec762caf04 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d3a13caf3b644632b250a0ec762caf04 2023-02-13 12:16:04,274 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3a13caf3b644632b250a0ec762caf04 2023-02-13 12:16:04,274 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d3a13caf3b644632b250a0ec762caf04, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 12:16:04,275 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 39ms, sequenceid=99, compaction requested=true 2023-02-13 12:16:04,275 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b4de9409ed4174da452e463dd5faeb66: 2023-02-13 12:16:04,275 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:04,275 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:16:04,275 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:16:04,275 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,275 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 12:16:04,275 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 12:16:04,283 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,283 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:04,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:04,284 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,284 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:16:04,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:04,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:04,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:04,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 12:16:04,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5433ec0[Count = 0] remaining members to acquire global barrier 2023-02-13 12:16:04,287 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 12:16:04,287 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,292 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,292 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,292 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,292 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:04,292 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,292 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:16:04,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,292 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:04,292 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:16:04,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,293 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:04,293 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 12:16:04,293 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:04,293 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:16:04,293 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 12:16:04,293 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:16:04,300 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,300 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:04,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:04,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:04,300 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:04,301 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:04,309 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:04,309 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:04,309 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:04,309 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 12:16:04,309 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:04,309 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 12:16:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:04,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:16:04,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:16:04,318 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 12:16:04,318 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:16:04,318 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:16:04,318 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,318 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:16:04,325 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,325 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,325 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,325 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,325 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,326 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:16:04,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:16:04,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,326 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,327 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:16:04,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,328 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,329 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:16:04,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,331 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:16:04,332 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,332 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,342 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:16:04,342 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 12:16:04,342 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:04,342 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:16:04,343 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:16:04,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,343 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:04,343 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:16:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:16:04,343 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,343 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:16:04,343 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,343 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,343 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:04,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:16:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:16:04,443 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:16:04,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:16:04,446 DEBUG [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,447 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:04,448 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:04,448 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:04,449 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:04,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b4de9409ed4174da452e463dd5faeb66, UNASSIGN}] 2023-02-13 12:16:04,451 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b4de9409ed4174da452e463dd5faeb66, UNASSIGN 2023-02-13 12:16:04,452 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=b4de9409ed4174da452e463dd5faeb66, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:04,452 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676290564452"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290564452"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290564452"}]},"ts":"1676290564452"} 2023-02-13 12:16:04,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure b4de9409ed4174da452e463dd5faeb66, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:16:04,611 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing b4de9409ed4174da452e463dd5faeb66, disabling compactions & flushes 2023-02-13 12:16:04,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:04,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:04,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. after waiting 0 ms 2023-02-13 12:16:04,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:04,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing b4de9409ed4174da452e463dd5faeb66 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-13 12:16:04,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/aef90e9726d14cae85ffbea860fd6d05 2023-02-13 12:16:04,625 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:16:04,630 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aef90e9726d14cae85ffbea860fd6d05 2023-02-13 12:16:04,642 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/fb6a8f6891574d24a15d348b7cf8a593 2023-02-13 12:16:04,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb6a8f6891574d24a15d348b7cf8a593 2023-02-13 12:16:04,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf1/aef90e9726d14cae85ffbea860fd6d05 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/aef90e9726d14cae85ffbea860fd6d05 2023-02-13 12:16:04,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aef90e9726d14cae85ffbea860fd6d05 2023-02-13 12:16:04,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/aef90e9726d14cae85ffbea860fd6d05, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 12:16:04,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.tmp/cf2/fb6a8f6891574d24a15d348b7cf8a593 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/fb6a8f6891574d24a15d348b7cf8a593 2023-02-13 12:16:04,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb6a8f6891574d24a15d348b7cf8a593 2023-02-13 12:16:04,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/fb6a8f6891574d24a15d348b7cf8a593, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 12:16:04,656 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 44ms, sequenceid=103, compaction requested=true 2023-02-13 12:16:04,665 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-13 12:16:04,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:16:04,667 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. 2023-02-13 12:16:04,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for b4de9409ed4174da452e463dd5faeb66: Waiting for close lock at 1676290564611Running coprocessor pre-close hooks at 1676290564611Disabling compacts and flushes for region at 1676290564611Disabling writes for close at 1676290564612 (+1 ms)Obtaining lock to block concurrent updates at 1676290564612Preparing flush snapshotting stores in b4de9409ed4174da452e463dd5faeb66 at 1676290564612Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676290564612Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66. at 1676290564613 (+1 ms)Flushing b4de9409ed4174da452e463dd5faeb66/cf1: creating writer at 1676290564613Flushing b4de9409ed4174da452e463dd5faeb66/cf1: appending metadata at 1676290564615 (+2 ms)Flushing b4de9409ed4174da452e463dd5faeb66/cf1: closing flushed file at 1676290564615Flushing b4de9409ed4174da452e463dd5faeb66/cf2: creating writer at 1676290564630 (+15 ms)Flushing b4de9409ed4174da452e463dd5faeb66/cf2: appending metadata at 1676290564633 (+3 ms)Flushing b4de9409ed4174da452e463dd5faeb66/cf2: closing flushed file at 1676290564633Flushing b4de9409ed4174da452e463dd5faeb66/cf1: reopening flushed file at 1676290564646 (+13 ms)Flushing b4de9409ed4174da452e463dd5faeb66/cf2: reopening flushed file at 1676290564651 (+5 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for b4de9409ed4174da452e463dd5faeb66 in 44ms, sequenceid=103, compaction requested=true at 1676290564656 (+5 ms)Writing region close event to WAL at 1676290564662 (+6 ms)Running coprocessor post-close hooks at 1676290564666 (+4 ms)Closed at 1676290564667 (+1 ms) 2023-02-13 12:16:04,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,669 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=b4de9409ed4174da452e463dd5faeb66, regionState=CLOSED 2023-02-13 12:16:04,669 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676290564669"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290564669"}]},"ts":"1676290564669"} 2023-02-13 12:16:04,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-13 12:16:04,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure b4de9409ed4174da452e463dd5faeb66, server=jenkins-hbase16.apache.org,33055,1676290533708 in 211 msec 2023-02-13 12:16:04,673 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-13 12:16:04,673 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b4de9409ed4174da452e463dd5faeb66, UNASSIGN in 221 msec 2023-02-13 12:16:04,688 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=b4de9409ed4174da452e463dd5faeb66, threads=16 2023-02-13 12:16:04,688 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6013eec329b94144aff522e8a6c824aa for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,690 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6ef545109dcf47e0b9383bce3d2bc0f3 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,690 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,690 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,690 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,690 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/b096fd687bae4b0d8ac5f950d1a105ab for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,690 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,690 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d3a13caf3b644632b250a0ec762caf04 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,692 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d76b5448f9034a0a83588157300f5dd7 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,696 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/fb6a8f6891574d24a15d348b7cf8a593 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,696 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/1f5893517ddf4a11807c732097dc7f74 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,704 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,704 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,704 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,704 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,705 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6ef1a73a8e0644b7a2cc946415ba5ea2 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,709 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6013eec329b94144aff522e8a6c824aa 2023-02-13 12:16:04,710 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3a13caf3b644632b250a0ec762caf04 2023-02-13 12:16:04,712 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2c5d0fb72224e17bbca80ffdf8c384e 2023-02-13 12:16:04,715 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/1f5893517ddf4a11807c732097dc7f74, top=false 2023-02-13 12:16:04,717 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/b096fd687bae4b0d8ac5f950d1a105ab, top=false 2023-02-13 12:16:04,719 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d76b5448f9034a0a83588157300f5dd7, top=false 2023-02-13 12:16:04,722 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6ef545109dcf47e0b9383bce3d2bc0f3, top=false 2023-02-13 12:16:04,727 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 304056b95f9f43c39c48a011cc38f134 2023-02-13 12:16:04,728 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ef1a73a8e0644b7a2cc946415ba5ea2 2023-02-13 12:16:04,730 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb6a8f6891574d24a15d348b7cf8a593 2023-02-13 12:16:04,731 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6013eec329b94144aff522e8a6c824aa 2023-02-13 12:16:04,731 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6013eec329b94144aff522e8a6c824aa, top=true 2023-02-13 12:16:04,733 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3a13caf3b644632b250a0ec762caf04 2023-02-13 12:16:04,734 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d3a13caf3b644632b250a0ec762caf04, top=true 2023-02-13 12:16:04,741 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb6a8f6891574d24a15d348b7cf8a593 2023-02-13 12:16:04,741 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/fb6a8f6891574d24a15d348b7cf8a593, top=true 2023-02-13 12:16:04,761 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ef1a73a8e0644b7a2cc946415ba5ea2 2023-02-13 12:16:04,761 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6ef1a73a8e0644b7a2cc946415ba5ea2, top=true 2023-02-13 12:16:04,788 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-b096fd687bae4b0d8ac5f950d1a105ab for child: 11f43b1e7c3649d8ac68d960818d7123, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,792 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6013eec329b94144aff522e8a6c824aa for child: b39af216aaba42ddb0ac2f5e3ecf0267, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,792 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6013eec329b94144aff522e8a6c824aa for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,792 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/aef90e9726d14cae85ffbea860fd6d05 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,795 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aef90e9726d14cae85ffbea860fd6d05 2023-02-13 12:16:04,796 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d76b5448f9034a0a83588157300f5dd7 for child: 11f43b1e7c3649d8ac68d960818d7123, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,797 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aef90e9726d14cae85ffbea860fd6d05 2023-02-13 12:16:04,798 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/aef90e9726d14cae85ffbea860fd6d05, top=true 2023-02-13 12:16:04,798 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d76b5448f9034a0a83588157300f5dd7 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,798 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/ca8b931880134e96ab7a5adfead3b64b for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,801 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/ca8b931880134e96ab7a5adfead3b64b, top=false 2023-02-13 12:16:04,804 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d3a13caf3b644632b250a0ec762caf04 for child: b39af216aaba42ddb0ac2f5e3ecf0267, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,804 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d3a13caf3b644632b250a0ec762caf04 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,804 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/d41689a539d843338802577ea77e7b02 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,804 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef545109dcf47e0b9383bce3d2bc0f3 for child: 11f43b1e7c3649d8ac68d960818d7123, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,804 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-1f5893517ddf4a11807c732097dc7f74 for child: 11f43b1e7c3649d8ac68d960818d7123, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,807 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41689a539d843338802577ea77e7b02 2023-02-13 12:16:04,807 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/1f5893517ddf4a11807c732097dc7f74 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,808 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/dffdf58afd9c40c49a26273b78a31699 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,811 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/dffdf58afd9c40c49a26273b78a31699, top=false 2023-02-13 12:16:04,821 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-fb6a8f6891574d24a15d348b7cf8a593 for child: b39af216aaba42ddb0ac2f5e3ecf0267, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,821 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/fb6a8f6891574d24a15d348b7cf8a593 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,821 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef1a73a8e0644b7a2cc946415ba5ea2 for child: b39af216aaba42ddb0ac2f5e3ecf0267, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,821 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6ef1a73a8e0644b7a2cc946415ba5ea2 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,822 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-aef90e9726d14cae85ffbea860fd6d05 for child: b39af216aaba42ddb0ac2f5e3ecf0267, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,822 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/aef90e9726d14cae85ffbea860fd6d05 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,844 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-dffdf58afd9c40c49a26273b78a31699 for child: 11f43b1e7c3649d8ac68d960818d7123, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,844 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-ca8b931880134e96ab7a5adfead3b64b for child: 11f43b1e7c3649d8ac68d960818d7123, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,854 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/dffdf58afd9c40c49a26273b78a31699 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,854 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/ca8b931880134e96ab7a5adfead3b64b for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,865 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41689a539d843338802577ea77e7b02 2023-02-13 12:16:04,865 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/d41689a539d843338802577ea77e7b02, top=true 2023-02-13 12:16:04,865 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2c5d0fb72224e17bbca80ffdf8c384e 2023-02-13 12:16:04,866 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6ef545109dcf47e0b9383bce3d2bc0f3 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,865 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/b096fd687bae4b0d8ac5f950d1a105ab for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,868 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 304056b95f9f43c39c48a011cc38f134 2023-02-13 12:16:04,904 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/.splits/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d41689a539d843338802577ea77e7b02 for child: b39af216aaba42ddb0ac2f5e3ecf0267, parent: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,904 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/d41689a539d843338802577ea77e7b02 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,904 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,920 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,921 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,924 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,924 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,928 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,932 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,934 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3 for region: b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:04,934 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region b4de9409ed4174da452e463dd5faeb66 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 12:16:04,967 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 12:16:04,969 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 12:16:04,970 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676290564970"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676290564970"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676290564970"}]},"ts":"1676290564970"} 2023-02-13 12:16:04,970 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676290564970"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290564970"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290564970"}]},"ts":"1676290564970"} 2023-02-13 12:16:04,971 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676290564970"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290564970"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290564970"}]},"ts":"1676290564970"} 2023-02-13 12:16:04,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=11f43b1e7c3649d8ac68d960818d7123, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b39af216aaba42ddb0ac2f5e3ecf0267, ASSIGN}] 2023-02-13 12:16:04,974 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b39af216aaba42ddb0ac2f5e3ecf0267, ASSIGN 2023-02-13 12:16:04,975 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b39af216aaba42ddb0ac2f5e3ecf0267, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:16:04,975 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=11f43b1e7c3649d8ac68d960818d7123, ASSIGN 2023-02-13 12:16:04,976 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=11f43b1e7c3649d8ac68d960818d7123, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:16:05,125 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 12:16:05,127 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=11f43b1e7c3649d8ac68d960818d7123, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:05,127 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676290565127"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290565127"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290565127"}]},"ts":"1676290565127"} 2023-02-13 12:16:05,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 11f43b1e7c3649d8ac68d960818d7123, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:16:05,133 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=b39af216aaba42ddb0ac2f5e3ecf0267, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:05,133 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676290565133"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290565133"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290565133"}]},"ts":"1676290565133"} 2023-02-13 12:16:05,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure b39af216aaba42ddb0ac2f5e3ecf0267, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:16:05,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:05,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b39af216aaba42ddb0ac2f5e3ecf0267, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.', STARTKEY => '60', ENDKEY => ''} 2023-02-13 12:16:05,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:16:05,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:05,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:16:05,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:05,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:05,291 INFO [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:05,292 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1 2023-02-13 12:16:05,292 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1 2023-02-13 12:16:05,293 INFO [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b39af216aaba42ddb0ac2f5e3ecf0267 columnFamilyName cf1 2023-02-13 12:16:05,308 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:05,310 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134-top 2023-02-13 12:16:05,314 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a-top 2023-02-13 12:16:05,319 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e-top 2023-02-13 12:16:05,322 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0-top 2023-02-13 12:16:05,326 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ef1a73a8e0644b7a2cc946415ba5ea2 2023-02-13 12:16:05,326 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef1a73a8e0644b7a2cc946415ba5ea2 2023-02-13 12:16:05,329 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aef90e9726d14cae85ffbea860fd6d05 2023-02-13 12:16:05,329 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-aef90e9726d14cae85ffbea860fd6d05 2023-02-13 12:16:05,333 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41689a539d843338802577ea77e7b02 2023-02-13 12:16:05,333 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d41689a539d843338802577ea77e7b02 2023-02-13 12:16:05,333 INFO [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(310): Store=b39af216aaba42ddb0ac2f5e3ecf0267/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:05,333 INFO [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:05,334 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2 2023-02-13 12:16:05,334 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2 2023-02-13 12:16:05,335 INFO [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b39af216aaba42ddb0ac2f5e3ecf0267 columnFamilyName cf2 2023-02-13 12:16:05,347 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3-top 2023-02-13 12:16:05,351 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:05,352 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e-top 2023-02-13 12:16:05,356 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640-top 2023-02-13 12:16:05,361 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c-top 2023-02-13 12:16:05,364 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6013eec329b94144aff522e8a6c824aa 2023-02-13 12:16:05,364 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6013eec329b94144aff522e8a6c824aa 2023-02-13 12:16:05,367 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3a13caf3b644632b250a0ec762caf04 2023-02-13 12:16:05,367 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d3a13caf3b644632b250a0ec762caf04 2023-02-13 12:16:05,372 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb6a8f6891574d24a15d348b7cf8a593 2023-02-13 12:16:05,372 DEBUG [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-fb6a8f6891574d24a15d348b7cf8a593 2023-02-13 12:16:05,372 INFO [StoreOpener-b39af216aaba42ddb0ac2f5e3ecf0267-1] regionserver.HStore(310): Store=b39af216aaba42ddb0ac2f5e3ecf0267/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:05,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:05,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:05,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:16:05,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:05,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened b39af216aaba42ddb0ac2f5e3ecf0267; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62258510, jitterRate=-0.07227590680122375}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:16:05,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for b39af216aaba42ddb0ac2f5e3ecf0267: Running coprocessor pre-open hook at 1676290565289Writing region info on filesystem at 1676290565289Initializing all the Stores at 1676290565291 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290565291Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290565291Cleaning up temporary data from old regions at 1676290565375 (+84 ms)Cleaning up detritus from prior splits at 1676290565375Running coprocessor post-open hooks at 1676290565378 (+3 ms)Region opened successfully at 1676290565378 2023-02-13 12:16:05,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267., pid=78, masterSystemTime=1676290565281 2023-02-13 12:16:05,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:16:05,379 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:16:05,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:16:05,379 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:16:05,380 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:05,380 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1912): b39af216aaba42ddb0ac2f5e3ecf0267/cf1 is initiating minor compaction (all files) 2023-02-13 12:16:05,380 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:05,380 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1912): b39af216aaba42ddb0ac2f5e3ecf0267/cf2 is initiating minor compaction (all files) 2023-02-13 12:16:05,380 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b39af216aaba42ddb0ac2f5e3ecf0267/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:05,380 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b39af216aaba42ddb0ac2f5e3ecf0267/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:05,380 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6013eec329b94144aff522e8a6c824aa, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d3a13caf3b644632b250a0ec762caf04, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-fb6a8f6891574d24a15d348b7cf8a593] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/.tmp, totalSize=34.8 K 2023-02-13 12:16:05,380 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d41689a539d843338802577ea77e7b02, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef1a73a8e0644b7a2cc946415ba5ea2, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-aef90e9726d14cae85ffbea860fd6d05] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/.tmp, totalSize=34.8 K 2023-02-13 12:16:05,381 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676290561596 2023-02-13 12:16:05,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:05,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:05,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:05,381 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=b39af216aaba42ddb0ac2f5e3ecf0267, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:05,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 11f43b1e7c3649d8ac68d960818d7123, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.', STARTKEY => '', ENDKEY => '60'} 2023-02-13 12:16:05,382 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676290565381"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290565381"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290565381"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290565381"}]},"ts":"1676290565381"} 2023-02-13 12:16:05,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:16:05,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:05,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:16:05,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:05,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:05,381 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676290561596 2023-02-13 12:16:05,381 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676290562314 2023-02-13 12:16:05,383 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676290562593 2023-02-13 12:16:05,384 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676290562314 2023-02-13 12:16:05,384 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676290563675 2023-02-13 12:16:05,384 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676290562593 2023-02-13 12:16:05,384 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6013eec329b94144aff522e8a6c824aa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 12:16:05,385 INFO [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:05,385 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676290563675 2023-02-13 12:16:05,385 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d3a13caf3b644632b250a0ec762caf04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 12:16:05,385 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d41689a539d843338802577ea77e7b02, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 12:16:05,386 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-fb6a8f6891574d24a15d348b7cf8a593, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 12:16:05,386 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1 2023-02-13 12:16:05,386 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef1a73a8e0644b7a2cc946415ba5ea2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 12:16:05,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-13 12:16:05,386 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1 2023-02-13 12:16:05,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure b39af216aaba42ddb0ac2f5e3ecf0267, server=jenkins-hbase16.apache.org,33055,1676290533708 in 249 msec 2023-02-13 12:16:05,386 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-aef90e9726d14cae85ffbea860fd6d05, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 12:16:05,387 INFO [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11f43b1e7c3649d8ac68d960818d7123 columnFamilyName cf1 2023-02-13 12:16:05,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b39af216aaba42ddb0ac2f5e3ecf0267, ASSIGN in 413 msec 2023-02-13 12:16:05,398 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:05,398 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134-bottom 2023-02-13 12:16:05,400 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] throttle.PressureAwareThroughputController(145): b39af216aaba42ddb0ac2f5e3ecf0267#cf2#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:16:05,401 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] throttle.PressureAwareThroughputController(145): b39af216aaba42ddb0ac2f5e3ecf0267#cf1#compaction#73 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:16:05,404 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a-bottom 2023-02-13 12:16:05,407 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e-bottom 2023-02-13 12:16:05,411 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0-bottom 2023-02-13 12:16:05,414 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-1f5893517ddf4a11807c732097dc7f74 2023-02-13 12:16:05,417 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-ca8b931880134e96ab7a5adfead3b64b 2023-02-13 12:16:05,422 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-dffdf58afd9c40c49a26273b78a31699 2023-02-13 12:16:05,428 INFO [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(310): Store=11f43b1e7c3649d8ac68d960818d7123/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:05,428 INFO [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:05,430 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2 2023-02-13 12:16:05,430 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2 2023-02-13 12:16:05,431 INFO [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11f43b1e7c3649d8ac68d960818d7123 columnFamilyName cf2 2023-02-13 12:16:05,434 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 419e6e6227c6498f849ffd2e0ba17b89: NONE, but ROW specified in column family configuration 2023-02-13 12:16:05,434 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 6131b1f88b7241618c35f5302fda4600: NONE, but ROW specified in column family configuration 2023-02-13 12:16:05,434 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/.tmp/cf2/419e6e6227c6498f849ffd2e0ba17b89 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/419e6e6227c6498f849ffd2e0ba17b89 2023-02-13 12:16:05,435 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/.tmp/cf1/6131b1f88b7241618c35f5302fda4600 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/6131b1f88b7241618c35f5302fda4600 2023-02-13 12:16:05,439 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 6131b1f88b7241618c35f5302fda4600: NONE, but ROW specified in column family configuration 2023-02-13 12:16:05,440 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b39af216aaba42ddb0ac2f5e3ecf0267/cf1 of b39af216aaba42ddb0ac2f5e3ecf0267 into 6131b1f88b7241618c35f5302fda4600(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:16:05,440 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 419e6e6227c6498f849ffd2e0ba17b89: NONE, but ROW specified in column family configuration 2023-02-13 12:16:05,440 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b39af216aaba42ddb0ac2f5e3ecf0267: 2023-02-13 12:16:05,440 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267., storeName=b39af216aaba42ddb0ac2f5e3ecf0267/cf1, priority=9, startTime=1676290565378; duration=0sec 2023-02-13 12:16:05,440 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:16:05,441 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b39af216aaba42ddb0ac2f5e3ecf0267/cf2 of b39af216aaba42ddb0ac2f5e3ecf0267 into 419e6e6227c6498f849ffd2e0ba17b89(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:16:05,441 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b39af216aaba42ddb0ac2f5e3ecf0267: 2023-02-13 12:16:05,441 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267., storeName=b39af216aaba42ddb0ac2f5e3ecf0267/cf2, priority=9, startTime=1676290565379; duration=0sec 2023-02-13 12:16:05,441 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:16:05,442 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3-bottom 2023-02-13 12:16:05,445 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:05,445 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e-bottom 2023-02-13 12:16:05,449 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640-bottom 2023-02-13 12:16:05,452 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c-bottom 2023-02-13 12:16:05,456 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef545109dcf47e0b9383bce3d2bc0f3 2023-02-13 12:16:05,460 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-b096fd687bae4b0d8ac5f950d1a105ab 2023-02-13 12:16:05,466 DEBUG [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d76b5448f9034a0a83588157300f5dd7 2023-02-13 12:16:05,466 INFO [StoreOpener-11f43b1e7c3649d8ac68d960818d7123-1] regionserver.HStore(310): Store=11f43b1e7c3649d8ac68d960818d7123/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:05,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:05,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:05,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:16:05,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:05,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 11f43b1e7c3649d8ac68d960818d7123; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63243841, jitterRate=-0.05759333074092865}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:16:05,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 11f43b1e7c3649d8ac68d960818d7123: Running coprocessor pre-open hook at 1676290565382Writing region info on filesystem at 1676290565382Initializing all the Stores at 1676290565384 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290565384Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290565384Cleaning up temporary data from old regions at 1676290565469 (+85 ms)Cleaning up detritus from prior splits at 1676290565469Running coprocessor post-open hooks at 1676290565471 (+2 ms)Region opened successfully at 1676290565471 2023-02-13 12:16:05,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123., pid=77, masterSystemTime=1676290565281 2023-02-13 12:16:05,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:16:05,473 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:16:05,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:16:05,473 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:16:05,473 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:05,473 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1912): 11f43b1e7c3649d8ac68d960818d7123/cf1 is initiating minor compaction (all files) 2023-02-13 12:16:05,473 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:05,474 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1912): 11f43b1e7c3649d8ac68d960818d7123/cf2 is initiating minor compaction (all files) 2023-02-13 12:16:05,474 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 11f43b1e7c3649d8ac68d960818d7123/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:05,474 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 11f43b1e7c3649d8ac68d960818d7123/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:05,474 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-dffdf58afd9c40c49a26273b78a31699, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-ca8b931880134e96ab7a5adfead3b64b, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-1f5893517ddf4a11807c732097dc7f74, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134-bottom] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/.tmp, totalSize=35.0 K 2023-02-13 12:16:05,474 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-b096fd687bae4b0d8ac5f950d1a105ab, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d76b5448f9034a0a83588157300f5dd7, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef545109dcf47e0b9383bce3d2bc0f3, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e-bottom] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/.tmp, totalSize=35.0 K 2023-02-13 12:16:05,475 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290561596 2023-02-13 12:16:05,475 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290561596 2023-02-13 12:16:05,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:05,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:05,475 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=11f43b1e7c3649d8ac68d960818d7123, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:05,475 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290562314 2023-02-13 12:16:05,476 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290562314 2023-02-13 12:16:05,476 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676290565475"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290565475"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290565475"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290565475"}]},"ts":"1676290565475"} 2023-02-13 12:16:05,476 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290562593 2023-02-13 12:16:05,476 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290562593 2023-02-13 12:16:05,476 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-dffdf58afd9c40c49a26273b78a31699, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290562888 2023-02-13 12:16:05,476 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-b096fd687bae4b0d8ac5f950d1a105ab, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290562888 2023-02-13 12:16:05,477 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-ca8b931880134e96ab7a5adfead3b64b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290563164 2023-02-13 12:16:05,477 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d76b5448f9034a0a83588157300f5dd7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290563164 2023-02-13 12:16:05,477 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-1f5893517ddf4a11807c732097dc7f74, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290563437 2023-02-13 12:16:05,477 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef545109dcf47e0b9383bce3d2bc0f3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290563437 2023-02-13 12:16:05,478 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676290563675 2023-02-13 12:16:05,478 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676290563675 2023-02-13 12:16:05,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-13 12:16:05,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 11f43b1e7c3649d8ac68d960818d7123, server=jenkins-hbase16.apache.org,33055,1676290533708 in 348 msec 2023-02-13 12:16:05,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-02-13 12:16:05,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=11f43b1e7c3649d8ac68d960818d7123, ASSIGN in 508 msec 2023-02-13 12:16:05,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 in 1.0360 sec 2023-02-13 12:16:05,496 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] throttle.PressureAwareThroughputController(145): 11f43b1e7c3649d8ac68d960818d7123#cf1#compaction#74 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:16:05,497 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] throttle.PressureAwareThroughputController(145): 11f43b1e7c3649d8ac68d960818d7123#cf2#compaction#75 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:16:05,517 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/.tmp/cf1/be5348e86ac7452eac2c5924809802c1 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/be5348e86ac7452eac2c5924809802c1 2023-02-13 12:16:05,518 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/.tmp/cf2/4a8f3d08e1934df58f6f1f26c3e576ac as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/4a8f3d08e1934df58f6f1f26c3e576ac 2023-02-13 12:16:05,522 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 11f43b1e7c3649d8ac68d960818d7123/cf1 of 11f43b1e7c3649d8ac68d960818d7123 into be5348e86ac7452eac2c5924809802c1(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:16:05,522 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 11f43b1e7c3649d8ac68d960818d7123: 2023-02-13 12:16:05,522 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 11f43b1e7c3649d8ac68d960818d7123/cf2 of 11f43b1e7c3649d8ac68d960818d7123 into 4a8f3d08e1934df58f6f1f26c3e576ac(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:16:05,522 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123., storeName=11f43b1e7c3649d8ac68d960818d7123/cf1, priority=9, startTime=1676290565472; duration=0sec 2023-02-13 12:16:05,522 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 11f43b1e7c3649d8ac68d960818d7123: 2023-02-13 12:16:05,522 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123., storeName=11f43b1e7c3649d8ac68d960818d7123/cf2, priority=9, startTime=1676290565473; duration=0sec 2023-02-13 12:16:05,522 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:16:05,522 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:16:05,724 DEBUG [Listener at localhost.localdomain/46087] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 12:16:05,728 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:57868, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 12:16:05,729 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33055] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:05,730 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33055] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 12:16:05,730 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33055] regionserver.HStore(1912): 11f43b1e7c3649d8ac68d960818d7123/cf1 is initiating major compaction (all files) 2023-02-13 12:16:05,730 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33055] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@65a8dae1; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:16:05,730 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 11f43b1e7c3649d8ac68d960818d7123/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:05,730 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33055] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 12:16:05,730 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/be5348e86ac7452eac2c5924809802c1] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/.tmp, totalSize=6.8 K 2023-02-13 12:16:05,730 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33055] regionserver.HStore(1912): 11f43b1e7c3649d8ac68d960818d7123/cf2 is initiating major compaction (all files) 2023-02-13 12:16:05,730 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33055] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@37a441dc; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:16:05,730 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 11f43b1e7c3649d8ac68d960818d7123/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:05,730 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/4a8f3d08e1934df58f6f1f26c3e576ac] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/.tmp, totalSize=6.8 K 2023-02-13 12:16:05,731 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting be5348e86ac7452eac2c5924809802c1, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676290561596 2023-02-13 12:16:05,731 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 4a8f3d08e1934df58f6f1f26c3e576ac, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676290561596 2023-02-13 12:16:05,732 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33055] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:05,732 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33055] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 12:16:05,732 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33055] regionserver.HStore(1912): b39af216aaba42ddb0ac2f5e3ecf0267/cf1 is initiating major compaction (all files) 2023-02-13 12:16:05,732 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33055] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7a7160f2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:16:05,732 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33055] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 12:16:05,733 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33055] regionserver.HStore(1912): b39af216aaba42ddb0ac2f5e3ecf0267/cf2 is initiating major compaction (all files) 2023-02-13 12:16:05,733 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33055] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3d3a9626; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 12:16:05,733 INFO [Listener at localhost.localdomain/46087] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 12:16:05,734 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] throttle.PressureAwareThroughputController(145): 11f43b1e7c3649d8ac68d960818d7123#cf1#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:16:05,734 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] throttle.PressureAwareThroughputController(145): 11f43b1e7c3649d8ac68d960818d7123#cf2#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:16:05,747 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33055] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. is not mergeable because it has references 2023-02-13 12:16:05,748 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33055] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. is not mergeable because it has references 2023-02-13 12:16:05,758 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/.tmp/cf2/00e80205168943f88a0001d3741f76ef as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/00e80205168943f88a0001d3741f76ef 2023-02-13 12:16:05,758 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/.tmp/cf1/55d0bf9e4f5342a899fd41f3b5398a54 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/55d0bf9e4f5342a899fd41f3b5398a54 2023-02-13 12:16:05,762 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 11f43b1e7c3649d8ac68d960818d7123/cf2 of 11f43b1e7c3649d8ac68d960818d7123 into 00e80205168943f88a0001d3741f76ef(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:16:05,762 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 11f43b1e7c3649d8ac68d960818d7123/cf1 of 11f43b1e7c3649d8ac68d960818d7123 into 55d0bf9e4f5342a899fd41f3b5398a54(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:16:05,762 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 11f43b1e7c3649d8ac68d960818d7123: 2023-02-13 12:16:05,762 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 11f43b1e7c3649d8ac68d960818d7123: 2023-02-13 12:16:05,764 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676290565730; duration=0sec 2023-02-13 12:16:05,764 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676290565730; duration=0sec 2023-02-13 12:16:05,764 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 12:16:05,764 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 12:16:05,764 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b39af216aaba42ddb0ac2f5e3ecf0267/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:05,764 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b39af216aaba42ddb0ac2f5e3ecf0267/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:05,764 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/6131b1f88b7241618c35f5302fda4600] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/.tmp, totalSize=5.0 K 2023-02-13 12:16:05,764 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/419e6e6227c6498f849ffd2e0ba17b89] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/.tmp, totalSize=5.0 K 2023-02-13 12:16:05,764 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 6131b1f88b7241618c35f5302fda4600, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 12:16:05,764 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 419e6e6227c6498f849ffd2e0ba17b89, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 12:16:05,767 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] throttle.PressureAwareThroughputController(145): b39af216aaba42ddb0ac2f5e3ecf0267#cf1#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:16:05,767 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] throttle.PressureAwareThroughputController(145): b39af216aaba42ddb0ac2f5e3ecf0267#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:16:05,790 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8e967c134f7d462ebab2c984b080e902: NONE, but ROW specified in column family configuration 2023-02-13 12:16:05,790 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/.tmp/cf2/8e967c134f7d462ebab2c984b080e902 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/8e967c134f7d462ebab2c984b080e902 2023-02-13 12:16:05,791 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for a6b9fd533ead489cb7569c35b2ca59e2: NONE, but ROW specified in column family configuration 2023-02-13 12:16:05,792 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/.tmp/cf1/a6b9fd533ead489cb7569c35b2ca59e2 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/a6b9fd533ead489cb7569c35b2ca59e2 2023-02-13 12:16:05,793 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8e967c134f7d462ebab2c984b080e902: NONE, but ROW specified in column family configuration 2023-02-13 12:16:05,794 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in b39af216aaba42ddb0ac2f5e3ecf0267/cf2 of b39af216aaba42ddb0ac2f5e3ecf0267 into 8e967c134f7d462ebab2c984b080e902(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:16:05,794 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b39af216aaba42ddb0ac2f5e3ecf0267: 2023-02-13 12:16:05,795 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676290565733; duration=0sec 2023-02-13 12:16:05,795 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:16:05,796 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for a6b9fd533ead489cb7569c35b2ca59e2: NONE, but ROW specified in column family configuration 2023-02-13 12:16:05,797 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in b39af216aaba42ddb0ac2f5e3ecf0267/cf1 of b39af216aaba42ddb0ac2f5e3ecf0267 into a6b9fd533ead489cb7569c35b2ca59e2(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:16:05,797 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b39af216aaba42ddb0ac2f5e3ecf0267: 2023-02-13 12:16:05,798 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676290565732; duration=0sec 2023-02-13 12:16:05,798 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:16:05,856 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33055] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. is not mergeable because it has references 2023-02-13 12:16:05,856 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33055] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. is not mergeable because it has references 2023-02-13 12:16:05,864 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-13 12:16:05,865 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:16:05,867 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:05,867 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:05,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:05,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 12:16:05,870 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290565870"}]},"ts":"1676290565870"} 2023-02-13 12:16:05,871 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-13 12:16:05,908 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-13 12:16:05,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=11f43b1e7c3649d8ac68d960818d7123, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b39af216aaba42ddb0ac2f5e3ecf0267, UNASSIGN}] 2023-02-13 12:16:05,911 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b39af216aaba42ddb0ac2f5e3ecf0267, UNASSIGN 2023-02-13 12:16:05,911 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=b39af216aaba42ddb0ac2f5e3ecf0267, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:05,912 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676290565911"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290565911"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290565911"}]},"ts":"1676290565911"} 2023-02-13 12:16:05,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure b39af216aaba42ddb0ac2f5e3ecf0267, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:16:05,916 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=11f43b1e7c3649d8ac68d960818d7123, UNASSIGN 2023-02-13 12:16:05,917 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=11f43b1e7c3649d8ac68d960818d7123, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:05,917 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676290565917"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290565917"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290565917"}]},"ts":"1676290565917"} 2023-02-13 12:16:05,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 11f43b1e7c3649d8ac68d960818d7123, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:16:05,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 12:16:06,066 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:06,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing b39af216aaba42ddb0ac2f5e3ecf0267, disabling compactions & flushes 2023-02-13 12:16:06,066 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:06,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:06,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. after waiting 0 ms 2023-02-13 12:16:06,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:06,067 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d41689a539d843338802577ea77e7b02, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef1a73a8e0644b7a2cc946415ba5ea2, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/6131b1f88b7241618c35f5302fda4600, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-aef90e9726d14cae85ffbea860fd6d05] to archive 2023-02-13 12:16:06,067 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:16:06,069 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,070 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,071 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,073 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,074 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d41689a539d843338802577ea77e7b02 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d41689a539d843338802577ea77e7b02 2023-02-13 12:16:06,076 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef1a73a8e0644b7a2cc946415ba5ea2 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef1a73a8e0644b7a2cc946415ba5ea2 2023-02-13 12:16:06,078 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/6131b1f88b7241618c35f5302fda4600 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/6131b1f88b7241618c35f5302fda4600 2023-02-13 12:16:06,079 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-aef90e9726d14cae85ffbea860fd6d05 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-aef90e9726d14cae85ffbea860fd6d05 2023-02-13 12:16:06,081 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6013eec329b94144aff522e8a6c824aa, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d3a13caf3b644632b250a0ec762caf04, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/419e6e6227c6498f849ffd2e0ba17b89, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-fb6a8f6891574d24a15d348b7cf8a593] to archive 2023-02-13 12:16:06,082 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:16:06,083 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,085 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,087 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,088 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,090 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6013eec329b94144aff522e8a6c824aa to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6013eec329b94144aff522e8a6c824aa 2023-02-13 12:16:06,092 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d3a13caf3b644632b250a0ec762caf04 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d3a13caf3b644632b250a0ec762caf04 2023-02-13 12:16:06,093 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/419e6e6227c6498f849ffd2e0ba17b89 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/419e6e6227c6498f849ffd2e0ba17b89 2023-02-13 12:16:06,095 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-fb6a8f6891574d24a15d348b7cf8a593 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-fb6a8f6891574d24a15d348b7cf8a593 2023-02-13 12:16:06,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 12:16:06,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:16:06,100 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267. 2023-02-13 12:16:06,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for b39af216aaba42ddb0ac2f5e3ecf0267: Waiting for close lock at 1676290566066Running coprocessor pre-close hooks at 1676290566066Disabling compacts and flushes for region at 1676290566066Disabling writes for close at 1676290566066Writing region close event to WAL at 1676290566096 (+30 ms)Running coprocessor post-close hooks at 1676290566100 (+4 ms)Closed at 1676290566100 2023-02-13 12:16:06,102 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:06,102 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:06,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 11f43b1e7c3649d8ac68d960818d7123, disabling compactions & flushes 2023-02-13 12:16:06,102 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:06,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:06,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. after waiting 0 ms 2023-02-13 12:16:06,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:06,102 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=b39af216aaba42ddb0ac2f5e3ecf0267, regionState=CLOSED 2023-02-13 12:16:06,102 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676290566102"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290566102"}]},"ts":"1676290566102"} 2023-02-13 12:16:06,104 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-dffdf58afd9c40c49a26273b78a31699, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-ca8b931880134e96ab7a5adfead3b64b, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-1f5893517ddf4a11807c732097dc7f74, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/be5348e86ac7452eac2c5924809802c1, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134-bottom] to archive 2023-02-13 12:16:06,105 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:16:06,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-13 12:16:06,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure b39af216aaba42ddb0ac2f5e3ecf0267, server=jenkins-hbase16.apache.org,33055,1676290533708 in 191 msec 2023-02-13 12:16:06,107 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/634f91df36ca46daa9608a034818fb6e.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b39af216aaba42ddb0ac2f5e3ecf0267, UNASSIGN in 197 msec 2023-02-13 12:16:06,108 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/3253afbef5f8450fafe3da95ddc9a73a.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,110 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/6b95bc15dab94af193614bf89dc1bdc0.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,111 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-dffdf58afd9c40c49a26273b78a31699 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-dffdf58afd9c40c49a26273b78a31699 2023-02-13 12:16:06,112 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-ca8b931880134e96ab7a5adfead3b64b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-ca8b931880134e96ab7a5adfead3b64b 2023-02-13 12:16:06,114 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-1f5893517ddf4a11807c732097dc7f74 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-1f5893517ddf4a11807c732097dc7f74 2023-02-13 12:16:06,115 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/be5348e86ac7452eac2c5924809802c1 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/be5348e86ac7452eac2c5924809802c1 2023-02-13 12:16:06,116 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/304056b95f9f43c39c48a011cc38f134.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,119 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-b096fd687bae4b0d8ac5f950d1a105ab, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d76b5448f9034a0a83588157300f5dd7, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef545109dcf47e0b9383bce3d2bc0f3, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/4a8f3d08e1934df58f6f1f26c3e576ac, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e-bottom] to archive 2023-02-13 12:16:06,119 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:16:06,121 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/ad90236225cd42c48c874fe838772640.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,123 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/9643bc3d56f3426fbfed9aad68002dc3.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,124 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/c89e30139b444d7f8b0317055459dd2c.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,126 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-b096fd687bae4b0d8ac5f950d1a105ab to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-b096fd687bae4b0d8ac5f950d1a105ab 2023-02-13 12:16:06,128 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d76b5448f9034a0a83588157300f5dd7 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-d76b5448f9034a0a83588157300f5dd7 2023-02-13 12:16:06,130 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef545109dcf47e0b9383bce3d2bc0f3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/testSplitTableRegionDeletedRowsDaughter=b4de9409ed4174da452e463dd5faeb66-6ef545109dcf47e0b9383bce3d2bc0f3 2023-02-13 12:16:06,131 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/4a8f3d08e1934df58f6f1f26c3e576ac to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/4a8f3d08e1934df58f6f1f26c3e576ac 2023-02-13 12:16:06,133 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/a2c5d0fb72224e17bbca80ffdf8c384e.b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 12:16:06,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:16:06,140 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123. 2023-02-13 12:16:06,140 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 11f43b1e7c3649d8ac68d960818d7123: Waiting for close lock at 1676290566102Running coprocessor pre-close hooks at 1676290566102Disabling compacts and flushes for region at 1676290566102Disabling writes for close at 1676290566102Writing region close event to WAL at 1676290566135 (+33 ms)Running coprocessor post-close hooks at 1676290566139 (+4 ms)Closed at 1676290566140 (+1 ms) 2023-02-13 12:16:06,142 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:06,143 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=11f43b1e7c3649d8ac68d960818d7123, regionState=CLOSED 2023-02-13 12:16:06,143 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676290566142"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290566142"}]},"ts":"1676290566142"} 2023-02-13 12:16:06,147 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-13 12:16:06,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 11f43b1e7c3649d8ac68d960818d7123, server=jenkins-hbase16.apache.org,33055,1676290533708 in 226 msec 2023-02-13 12:16:06,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-13 12:16:06,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=11f43b1e7c3649d8ac68d960818d7123, UNASSIGN in 239 msec 2023-02-13 12:16:06,151 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290566151"}]},"ts":"1676290566151"} 2023-02-13 12:16:06,152 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-13 12:16:06,158 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-13 12:16:06,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 292 msec 2023-02-13 12:16:06,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 12:16:06,173 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-13 12:16:06,174 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:06,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:06,178 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:06,179 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:06,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 12:16:06,186 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,186 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:06,186 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:06,189 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/recovered.edits] 2023-02-13 12:16:06,190 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/recovered.edits] 2023-02-13 12:16:06,190 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/recovered.edits] 2023-02-13 12:16:06,194 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/a6b9fd533ead489cb7569c35b2ca59e2 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf1/a6b9fd533ead489cb7569c35b2ca59e2 2023-02-13 12:16:06,194 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/1f5893517ddf4a11807c732097dc7f74 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/1f5893517ddf4a11807c732097dc7f74 2023-02-13 12:16:06,194 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/55d0bf9e4f5342a899fd41f3b5398a54 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf1/55d0bf9e4f5342a899fd41f3b5398a54 2023-02-13 12:16:06,196 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/304056b95f9f43c39c48a011cc38f134 2023-02-13 12:16:06,197 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/8e967c134f7d462ebab2c984b080e902 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/cf2/8e967c134f7d462ebab2c984b080e902 2023-02-13 12:16:06,198 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/00e80205168943f88a0001d3741f76ef to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/cf2/00e80205168943f88a0001d3741f76ef 2023-02-13 12:16:06,198 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/3253afbef5f8450fafe3da95ddc9a73a 2023-02-13 12:16:06,200 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/634f91df36ca46daa9608a034818fb6e 2023-02-13 12:16:06,200 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/recovered.edits/113.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267/recovered.edits/113.seqid 2023-02-13 12:16:06,201 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/recovered.edits/113.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123/recovered.edits/113.seqid 2023-02-13 12:16:06,201 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:16:06,201 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6b95bc15dab94af193614bf89dc1bdc0 2023-02-13 12:16:06,201 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/11f43b1e7c3649d8ac68d960818d7123 2023-02-13 12:16:06,203 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6ef1a73a8e0644b7a2cc946415ba5ea2 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/6ef1a73a8e0644b7a2cc946415ba5ea2 2023-02-13 12:16:06,203 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/aef90e9726d14cae85ffbea860fd6d05 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/aef90e9726d14cae85ffbea860fd6d05 2023-02-13 12:16:06,205 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/ca8b931880134e96ab7a5adfead3b64b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/ca8b931880134e96ab7a5adfead3b64b 2023-02-13 12:16:06,206 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/d41689a539d843338802577ea77e7b02 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/d41689a539d843338802577ea77e7b02 2023-02-13 12:16:06,207 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/dffdf58afd9c40c49a26273b78a31699 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf1/dffdf58afd9c40c49a26273b78a31699 2023-02-13 12:16:06,209 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6013eec329b94144aff522e8a6c824aa to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6013eec329b94144aff522e8a6c824aa 2023-02-13 12:16:06,210 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6ef545109dcf47e0b9383bce3d2bc0f3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/6ef545109dcf47e0b9383bce3d2bc0f3 2023-02-13 12:16:06,211 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/9643bc3d56f3426fbfed9aad68002dc3 2023-02-13 12:16:06,212 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/a2c5d0fb72224e17bbca80ffdf8c384e 2023-02-13 12:16:06,213 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/ad90236225cd42c48c874fe838772640 2023-02-13 12:16:06,214 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/b096fd687bae4b0d8ac5f950d1a105ab to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/b096fd687bae4b0d8ac5f950d1a105ab 2023-02-13 12:16:06,215 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/c89e30139b444d7f8b0317055459dd2c 2023-02-13 12:16:06,216 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d3a13caf3b644632b250a0ec762caf04 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d3a13caf3b644632b250a0ec762caf04 2023-02-13 12:16:06,217 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d76b5448f9034a0a83588157300f5dd7 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/d76b5448f9034a0a83588157300f5dd7 2023-02-13 12:16:06,218 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/fb6a8f6891574d24a15d348b7cf8a593 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/cf2/fb6a8f6891574d24a15d348b7cf8a593 2023-02-13 12:16:06,221 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/recovered.edits/106.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66/recovered.edits/106.seqid 2023-02-13 12:16:06,221 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b4de9409ed4174da452e463dd5faeb66 2023-02-13 12:16:06,221 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 12:16:06,223 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:06,225 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-13 12:16:06,227 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-13 12:16:06,228 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:06,228 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-13 12:16:06,228 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290566228"}]},"ts":"9223372036854775807"} 2023-02-13 12:16:06,228 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290566228"}]},"ts":"9223372036854775807"} 2023-02-13 12:16:06,228 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290566228"}]},"ts":"9223372036854775807"} 2023-02-13 12:16:06,229 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 12:16:06,229 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b4de9409ed4174da452e463dd5faeb66, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676290561093.b4de9409ed4174da452e463dd5faeb66.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 11f43b1e7c3649d8ac68d960818d7123, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676290564447.11f43b1e7c3649d8ac68d960818d7123.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => b39af216aaba42ddb0ac2f5e3ecf0267, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676290564447.b39af216aaba42ddb0ac2f5e3ecf0267.', STARTKEY => '60', ENDKEY => ''}] 2023-02-13 12:16:06,230 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-13 12:16:06,230 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676290566230"}]},"ts":"9223372036854775807"} 2023-02-13 12:16:06,231 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-13 12:16:06,240 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:16:06,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 66 msec 2023-02-13 12:16:06,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 12:16:06,281 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-13 12:16:06,295 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=413 (was 408) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:59312 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:59364 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:59456 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:59380 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:36320 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:42812 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:59336 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:59298 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:35982 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:50652 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3fbde3a3-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:36100 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:59388 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:50664 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:50644 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:59402 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1742415765_15 at /127.0.0.1:36030 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=856 (was 849) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=412 (was 335) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=5211 (was 5368) 2023-02-13 12:16:06,307 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=413, OpenFileDescriptor=856, MaxFileDescriptor=60000, SystemLoadAverage=412, ProcessCount=167, AvailableMemoryMB=5210 2023-02-13 12:16:06,308 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:16:06,317 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 12:16:06,317 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 12:16:06,318 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:16:06,320 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:16:06,321 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:16:06,322 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:06,323 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a empty. 2023-02-13 12:16:06,323 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:06,323 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 12:16:06,333 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 12:16:06,333 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 223dadc96dff0acc6b1c2fb058f22e1a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:16:06,341 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:16:06,341 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 223dadc96dff0acc6b1c2fb058f22e1a, disabling compactions & flushes 2023-02-13 12:16:06,341 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:06,341 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:06,341 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. after waiting 0 ms 2023-02-13 12:16:06,341 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:06,341 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:06,341 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 223dadc96dff0acc6b1c2fb058f22e1a: Waiting for close lock at 1676290566341Disabling compacts and flushes for region at 1676290566341Disabling writes for close at 1676290566341Writing region close event to WAL at 1676290566341Closed at 1676290566341 2023-02-13 12:16:06,343 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:16:06,343 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290566343"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290566343"}]},"ts":"1676290566343"} 2023-02-13 12:16:06,345 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:16:06,345 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:16:06,346 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290566345"}]},"ts":"1676290566345"} 2023-02-13 12:16:06,346 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-13 12:16:06,397 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:16:06,398 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:16:06,398 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:16:06,398 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:16:06,398 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:16:06,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, ASSIGN}] 2023-02-13 12:16:06,400 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, ASSIGN 2023-02-13 12:16:06,401 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,45345,1676290533799; forceNewPlan=false, retain=false 2023-02-13 12:16:06,551 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:16:06,555 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=223dadc96dff0acc6b1c2fb058f22e1a, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:06,555 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290566555"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290566555"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290566555"}]},"ts":"1676290566555"} 2023-02-13 12:16:06,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 223dadc96dff0acc6b1c2fb058f22e1a, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:16:06,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:06,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 223dadc96dff0acc6b1c2fb058f22e1a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:16:06,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:16:06,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:06,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:16:06,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:06,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:06,721 INFO [StoreOpener-223dadc96dff0acc6b1c2fb058f22e1a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:06,723 DEBUG [StoreOpener-223dadc96dff0acc6b1c2fb058f22e1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/cf1 2023-02-13 12:16:06,723 DEBUG [StoreOpener-223dadc96dff0acc6b1c2fb058f22e1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/cf1 2023-02-13 12:16:06,724 INFO [StoreOpener-223dadc96dff0acc6b1c2fb058f22e1a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 223dadc96dff0acc6b1c2fb058f22e1a columnFamilyName cf1 2023-02-13 12:16:06,724 INFO [StoreOpener-223dadc96dff0acc6b1c2fb058f22e1a-1] regionserver.HStore(310): Store=223dadc96dff0acc6b1c2fb058f22e1a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:06,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:06,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:06,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:06,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:16:06,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 223dadc96dff0acc6b1c2fb058f22e1a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69770056, jitterRate=0.03965485095977783}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:16:06,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 223dadc96dff0acc6b1c2fb058f22e1a: Running coprocessor pre-open hook at 1676290566719Writing region info on filesystem at 1676290566719Initializing all the Stores at 1676290566721 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290566721Cleaning up temporary data from old regions at 1676290566726 (+5 ms)Cleaning up detritus from prior splits at 1676290566727 (+1 ms)Running coprocessor post-open hooks at 1676290566731 (+4 ms)Region opened successfully at 1676290566731 2023-02-13 12:16:06,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a., pid=87, masterSystemTime=1676290566711 2023-02-13 12:16:06,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:06,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:06,735 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=223dadc96dff0acc6b1c2fb058f22e1a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:06,735 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290566735"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290566735"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290566735"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290566735"}]},"ts":"1676290566735"} 2023-02-13 12:16:06,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-13 12:16:06,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 223dadc96dff0acc6b1c2fb058f22e1a, server=jenkins-hbase16.apache.org,45345,1676290533799 in 179 msec 2023-02-13 12:16:06,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-13 12:16:06,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, ASSIGN in 338 msec 2023-02-13 12:16:06,739 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:16:06,739 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290566739"}]},"ts":"1676290566739"} 2023-02-13 12:16:06,740 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-13 12:16:06,776 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:16:06,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 458 msec 2023-02-13 12:16:06,831 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$12(2441): Client=jenkins//188.40.62.62 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 12:16:06,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:16:06,855 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 12:16:06,855 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 12:16:06,856 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 12:16:06,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-13 12:16:06,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, REOPEN/MOVE}] 2023-02-13 12:16:06,872 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, REOPEN/MOVE 2023-02-13 12:16:06,872 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=223dadc96dff0acc6b1c2fb058f22e1a, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:06,872 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290566872"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290566872"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290566872"}]},"ts":"1676290566872"} 2023-02-13 12:16:06,874 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 223dadc96dff0acc6b1c2fb058f22e1a, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:16:07,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:07,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 223dadc96dff0acc6b1c2fb058f22e1a, disabling compactions & flushes 2023-02-13 12:16:07,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:07,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:07,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. after waiting 0 ms 2023-02-13 12:16:07,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:07,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 12:16:07,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:16:07,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:07,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 223dadc96dff0acc6b1c2fb058f22e1a: Waiting for close lock at 1676290567027Running coprocessor pre-close hooks at 1676290567027Disabling compacts and flushes for region at 1676290567027Disabling writes for close at 1676290567029 (+2 ms)Writing region close event to WAL at 1676290567030 (+1 ms)Running coprocessor post-close hooks at 1676290567036 (+6 ms)Closed at 1676290567036 2023-02-13 12:16:07,036 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 223dadc96dff0acc6b1c2fb058f22e1a to self. 2023-02-13 12:16:07,038 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:07,039 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=223dadc96dff0acc6b1c2fb058f22e1a, regionState=CLOSED 2023-02-13 12:16:07,039 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290567039"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290567039"}]},"ts":"1676290567039"} 2023-02-13 12:16:07,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-13 12:16:07,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 223dadc96dff0acc6b1c2fb058f22e1a, server=jenkins-hbase16.apache.org,45345,1676290533799 in 166 msec 2023-02-13 12:16:07,044 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,45345,1676290533799; forceNewPlan=false, retain=true 2023-02-13 12:16:07,194 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:16:07,194 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=223dadc96dff0acc6b1c2fb058f22e1a, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:07,194 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290567194"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290567194"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290567194"}]},"ts":"1676290567194"} 2023-02-13 12:16:07,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 223dadc96dff0acc6b1c2fb058f22e1a, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:16:07,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:07,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 223dadc96dff0acc6b1c2fb058f22e1a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:16:07,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:16:07,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:07,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:16:07,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:07,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:07,353 INFO [StoreOpener-223dadc96dff0acc6b1c2fb058f22e1a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:07,354 DEBUG [StoreOpener-223dadc96dff0acc6b1c2fb058f22e1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/cf1 2023-02-13 12:16:07,354 DEBUG [StoreOpener-223dadc96dff0acc6b1c2fb058f22e1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/cf1 2023-02-13 12:16:07,355 INFO [StoreOpener-223dadc96dff0acc6b1c2fb058f22e1a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 223dadc96dff0acc6b1c2fb058f22e1a columnFamilyName cf1 2023-02-13 12:16:07,355 INFO [StoreOpener-223dadc96dff0acc6b1c2fb058f22e1a-1] regionserver.HStore(310): Store=223dadc96dff0acc6b1c2fb058f22e1a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:16:07,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:07,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:07,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:16:07,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 223dadc96dff0acc6b1c2fb058f22e1a; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75349542, jitterRate=0.12279567122459412}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:16:07,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 223dadc96dff0acc6b1c2fb058f22e1a: Running coprocessor pre-open hook at 1676290567352Writing region info on filesystem at 1676290567352Initializing all the Stores at 1676290567353 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290567353Cleaning up temporary data from old regions at 1676290567357 (+4 ms)Cleaning up detritus from prior splits at 1676290567357Running coprocessor post-open hooks at 1676290567360 (+3 ms)Region opened successfully at 1676290567360 2023-02-13 12:16:07,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a., pid=92, masterSystemTime=1676290567349 2023-02-13 12:16:07,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:07,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:16:07,363 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=223dadc96dff0acc6b1c2fb058f22e1a, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:16:07,363 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290567362"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290567362"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290567362"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290567362"}]},"ts":"1676290567362"} 2023-02-13 12:16:07,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-13 12:16:07,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 223dadc96dff0acc6b1c2fb058f22e1a, server=jenkins-hbase16.apache.org,45345,1676290533799 in 168 msec 2023-02-13 12:16:07,367 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-13 12:16:07,367 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, REOPEN/MOVE in 495 msec 2023-02-13 12:16:07,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-13 12:16:07,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 510 msec 2023-02-13 12:16:07,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517, ASSIGN}] 2023-02-13 12:16:07,375 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517, ASSIGN 2023-02-13 12:16:07,375 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 12:16:07,526 DEBUG [jenkins-hbase16:39699] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:16:07,527 DEBUG [jenkins-hbase16:39699] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:16:07,527 DEBUG [jenkins-hbase16:39699] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:16:07,527 DEBUG [jenkins-hbase16:39699] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:16:07,527 DEBUG [jenkins-hbase16:39699] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-13 12:16:07,529 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=07406c18ee535b7620fbe7fdd2bbe517, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:16:07,529 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290567529"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1676290567529"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676290567529"}]},"ts":"1676290567529"} 2023-02-13 12:16:07,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 07406c18ee535b7620fbe7fdd2bbe517, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:16:07,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517. 2023-02-13 12:16:07,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 07406c18ee535b7620fbe7fdd2bbe517, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-13 12:16:07,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:16:07,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 07406c18ee535b7620fbe7fdd2bbe517 2023-02-13 12:16:07,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:16:07,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 07406c18ee535b7620fbe7fdd2bbe517 2023-02-13 12:16:07,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 07406c18ee535b7620fbe7fdd2bbe517 2023-02-13 12:16:07,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:07,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:07,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:07,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:08,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:08,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:08,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:08,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:08,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:08,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:08,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:08,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:08,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:08,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:09,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:09,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:09,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:09,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:09,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:09,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:09,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:09,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:09,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:09,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:10,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:10,168 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:16:10,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:10,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:10,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:10,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:10,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:10,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:10,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:10,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:10,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:11,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:11,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:11,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:11,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:11,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:11,556 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-13 12:16:11,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:11,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:11,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:11,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:12,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:12,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:12,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:12,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:12,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:12,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:12,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:12,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:12,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:12,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:13,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:13,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:13,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:13,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:13,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:13,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:13,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:13,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:13,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:13,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:14,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:14,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:14,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:14,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:14,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:14,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:14,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:14,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:14,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:14,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:15,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:15,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:15,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:15,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:15,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:15,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:15,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:15,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:15,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:15,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:16,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:16,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:16,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:16,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:16,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:16,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:16,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:16,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:16,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:16,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:17,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:17,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:17,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:17,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:17,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:17,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:17,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:17,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:17,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:17,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:18,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:18,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:18,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:18,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:18,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:18,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:18,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:18,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:18,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:18,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:19,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:19,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:19,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:19,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:19,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:19,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:19,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:19,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:19,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:19,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:20,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:20,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:20,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:20,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:20,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:20,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:20,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:20,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:20,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:20,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:21,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:21,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:21,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:21,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:21,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:21,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:21,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:21,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:21,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:21,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:22,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:22,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:22,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:22,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:22,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:22,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:22,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:22,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:22,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:22,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:23,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:23,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:23,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:23,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:23,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:23,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:23,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:23,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:23,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:23,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:24,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:24,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:24,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:24,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:24,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:24,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:24,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:24,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:24,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:24,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:25,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:25,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:25,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:25,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:25,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:25,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:25,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:25,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:25,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:25,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:26,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:26,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:26,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:26,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:26,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:26,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:26,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:26,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:26,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:26,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:27,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:27,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:27,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:27,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:27,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:27,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:27,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:27,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:27,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:27,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:28,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:28,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:28,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:28,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:28,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:28,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:28,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:28,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:28,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:28,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:29,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:29,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:29,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:29,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:29,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:29,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:29,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:29,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:29,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:29,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:30,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:30,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:30,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:30,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:30,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:30,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:30,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:30,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:30,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:30,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:31,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:31,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:31,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:31,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:31,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:31,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:31,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:31,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:31,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:31,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:32,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:32,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:32,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:32,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:32,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:32,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:32,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:32,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:32,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:32,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:33,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:33,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:33,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:33,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:33,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:33,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:33,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:33,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:33,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:33,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:34,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:34,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:34,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:34,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:34,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:34,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:34,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:34,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:34,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:34,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:35,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:35,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:35,183 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 12:16:35,215 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 12:16:35,227 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 12:16:35,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:35,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:35,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:35,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:35,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:35,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:35,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:35,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:36,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:36,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:36,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:36,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:36,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:36,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:36,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:36,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:36,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:36,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:37,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:37,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:37,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:37,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:37,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:37,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:37,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:37,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:37,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:37,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:38,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:38,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:38,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:38,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:38,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:38,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:38,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:38,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:38,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:38,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:39,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:39,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:39,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:39,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:39,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:39,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:39,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:39,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:39,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:39,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:40,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:40,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:40,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:40,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:40,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:40,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:40,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:40,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:40,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:40,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:41,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:41,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:41,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:41,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:41,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:41,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:41,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:41,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:41,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:41,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:42,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:42,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:42,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:42,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:42,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:42,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:42,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:42,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:42,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:42,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:43,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:43,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:43,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:43,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:43,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:43,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:43,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:43,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:43,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:43,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:44,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:44,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:44,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:44,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:44,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:44,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:44,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:44,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:44,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:44,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:45,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:45,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:45,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:45,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:45,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:45,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:45,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:45,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:45,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:45,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:46,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:46,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:46,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:46,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:46,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:46,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:46,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:46,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:46,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:46,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:47,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:47,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:47,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:47,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:47,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:47,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:47,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:47,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:47,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:47,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:48,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:48,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:48,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:48,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:48,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:48,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:48,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:48,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:48,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:48,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:49,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:49,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:49,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:49,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:49,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:49,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:49,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:49,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:49,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:49,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:50,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:50,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:50,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:50,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:50,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:50,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:50,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:50,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:50,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:50,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:51,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:51,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:51,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:51,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:51,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:51,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:51,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:51,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:51,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:51,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:52,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:52,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:52,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:52,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:52,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:52,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:52,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:52,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:52,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:52,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:53,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:53,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:53,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:53,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:53,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:53,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:53,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:53,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:53,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:53,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:54,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:54,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:54,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:54,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:54,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:54,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:54,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:54,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:54,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:54,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:55,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:55,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:55,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:55,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:55,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:55,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:55,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:55,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:55,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:55,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:56,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:56,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:56,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:56,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:56,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:56,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:56,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:56,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:56,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:56,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:57,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:57,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:57,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:57,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:57,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:57,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:57,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:57,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:57,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:57,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:58,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:58,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:58,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:58,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:58,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:58,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:58,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:58,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:58,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:58,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:59,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:59,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:59,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:59,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:59,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:59,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:59,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:59,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:59,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:16:59,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:00,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:00,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:00,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:00,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:00,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:00,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:00,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:00,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:00,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:00,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:01,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:01,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:01,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:01,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:01,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:01,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:01,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:01,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:01,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:02,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:02,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:02,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:02,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:02,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:02,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:02,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:02,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:02,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:02,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:03,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:03,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:03,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:03,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:03,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:03,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:03,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:03,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:03,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:03,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:04,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:04,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:04,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:04,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:04,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:04,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:04,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:04,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:04,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:04,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:05,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:05,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:05,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:05,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:05,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:05,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:05,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:05,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:05,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:05,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:06,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 12:17:07,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:07,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 12:17:07,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:07,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:07,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 12:17:07,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:07,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:07,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:07,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 12:17:07,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:07,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:07,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:07,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:08,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:08,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 12:17:08,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:08,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:08,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:08,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:08,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:08,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:08,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:08,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:08,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:09,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:09,054 DEBUG [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:09,055 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:09,055 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:09,055 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:09,055 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:09,057 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 223dadc96dff0acc6b1c2fb058f22e1a, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:09,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 exec-time=5 msec 2023-02-13 12:17:09,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:09,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 12:17:09,315 INFO [StoreOpener-07406c18ee535b7620fbe7fdd2bbe517-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 07406c18ee535b7620fbe7fdd2bbe517 2023-02-13 12:17:09,318 DEBUG [StoreOpener-07406c18ee535b7620fbe7fdd2bbe517-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/cf1 2023-02-13 12:17:09,318 DEBUG [StoreOpener-07406c18ee535b7620fbe7fdd2bbe517-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/cf1 2023-02-13 12:17:09,318 INFO [StoreOpener-07406c18ee535b7620fbe7fdd2bbe517-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 07406c18ee535b7620fbe7fdd2bbe517 columnFamilyName cf1 2023-02-13 12:17:09,319 INFO [StoreOpener-07406c18ee535b7620fbe7fdd2bbe517-1] regionserver.HStore(310): Store=07406c18ee535b7620fbe7fdd2bbe517/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:09,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 07406c18ee535b7620fbe7fdd2bbe517; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72854635, jitterRate=0.08561865985393524}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:17:09,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 07406c18ee535b7620fbe7fdd2bbe517: Running coprocessor pre-open hook at 1676290567689Initializing all the Stores at 1676290629315 (+61626 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290629315Running coprocessor post-open hooks at 1676290629321 (+6 ms)Region opened successfully at 1676290629321 2023-02-13 12:17:09,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517., pid=94, masterSystemTime=1676290567685 2023-02-13 12:17:09,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517. 2023-02-13 12:17:09,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517. 2023-02-13 12:17:09,322 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=07406c18ee535b7620fbe7fdd2bbe517, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:09,323 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290629322"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1676290629322"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676290629322"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676290629322"}]},"ts":"1676290629322"} 2023-02-13 12:17:09,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-13 12:17:09,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 07406c18ee535b7620fbe7fdd2bbe517, server=jenkins-hbase16.apache.org,33055,1676290533708 in 1 mins, 1.793 sec 2023-02-13 12:17:09,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-13 12:17:09,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517, ASSIGN in 1 mins, 1.953 sec 2023-02-13 12:17:09,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.494 sec 2023-02-13 12:17:09,330 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676290566829","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62500,"client":"188.40.62.62:40934","queuetimems":0,"class":"HMaster"} 2023-02-13 12:17:09,330 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 605, skipping over this response of 3 bytes 2023-02-13 12:17:11,305 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up 2023-02-13 12:17:11,307 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:17:11,308 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:11,309 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:11,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:11,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 12:17:11,312 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290631312"}]},"ts":"1676290631312"} 2023-02-13 12:17:11,313 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-13 12:17:11,355 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-13 12:17:11,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517, UNASSIGN}] 2023-02-13 12:17:11,357 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517, UNASSIGN 2023-02-13 12:17:11,358 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=07406c18ee535b7620fbe7fdd2bbe517, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:11,358 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290631358"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1676290631358"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676290631358"}]},"ts":"1676290631358"} 2023-02-13 12:17:11,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 07406c18ee535b7620fbe7fdd2bbe517, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:11,361 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, UNASSIGN 2023-02-13 12:17:11,362 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=223dadc96dff0acc6b1c2fb058f22e1a, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:11,362 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290631361"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290631361"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290631361"}]},"ts":"1676290631361"} 2023-02-13 12:17:11,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 223dadc96dff0acc6b1c2fb058f22e1a, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:17:11,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 12:17:11,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 07406c18ee535b7620fbe7fdd2bbe517 2023-02-13 12:17:11,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 07406c18ee535b7620fbe7fdd2bbe517, disabling compactions & flushes 2023-02-13 12:17:11,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517. 2023-02-13 12:17:11,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517. 2023-02-13 12:17:11,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517. after waiting 0 ms 2023-02-13 12:17:11,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517. 2023-02-13 12:17:11,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:11,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517. 2023-02-13 12:17:11,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 07406c18ee535b7620fbe7fdd2bbe517: Waiting for close lock at 1676290631511Running coprocessor pre-close hooks at 1676290631511Disabling compacts and flushes for region at 1676290631511Disabling writes for close at 1676290631512 (+1 ms)Writing region close event to WAL at 1676290631513 (+1 ms)Running coprocessor post-close hooks at 1676290631513Closed at 1676290631513 2023-02-13 12:17:11,515 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 07406c18ee535b7620fbe7fdd2bbe517 2023-02-13 12:17:11,515 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:17:11,515 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=07406c18ee535b7620fbe7fdd2bbe517, regionState=CLOSED 2023-02-13 12:17:11,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 223dadc96dff0acc6b1c2fb058f22e1a, disabling compactions & flushes 2023-02-13 12:17:11,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:17:11,516 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290631515"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676290631515"}]},"ts":"1676290631515"} 2023-02-13 12:17:11,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:17:11,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. after waiting 0 ms 2023-02-13 12:17:11,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:17:11,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-13 12:17:11,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 07406c18ee535b7620fbe7fdd2bbe517, server=jenkins-hbase16.apache.org,33055,1676290533708 in 159 msec 2023-02-13 12:17:11,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 12:17:11,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:11,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a. 2023-02-13 12:17:11,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 223dadc96dff0acc6b1c2fb058f22e1a: Waiting for close lock at 1676290631515Running coprocessor pre-close hooks at 1676290631515Disabling compacts and flushes for region at 1676290631515Disabling writes for close at 1676290631516 (+1 ms)Writing region close event to WAL at 1676290631516Running coprocessor post-close hooks at 1676290631523 (+7 ms)Closed at 1676290631524 (+1 ms) 2023-02-13 12:17:11,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517, UNASSIGN in 166 msec 2023-02-13 12:17:11,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:17:11,525 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=223dadc96dff0acc6b1c2fb058f22e1a, regionState=CLOSED 2023-02-13 12:17:11,525 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676290631525"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290631525"}]},"ts":"1676290631525"} 2023-02-13 12:17:11,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-13 12:17:11,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 223dadc96dff0acc6b1c2fb058f22e1a, server=jenkins-hbase16.apache.org,45345,1676290533799 in 163 msec 2023-02-13 12:17:11,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-13 12:17:11,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=223dadc96dff0acc6b1c2fb058f22e1a, UNASSIGN in 172 msec 2023-02-13 12:17:11,530 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290631530"}]},"ts":"1676290631530"} 2023-02-13 12:17:11,531 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-13 12:17:11,539 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-13 12:17:11,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 231 msec 2023-02-13 12:17:11,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 12:17:11,615 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-13 12:17:11,615 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:11,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:11,618 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:11,618 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:11,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 12:17:11,624 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:17:11,626 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/recovered.edits] 2023-02-13 12:17:11,631 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/recovered.edits/7.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a/recovered.edits/7.seqid 2023-02-13 12:17:11,631 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/223dadc96dff0acc6b1c2fb058f22e1a 2023-02-13 12:17:11,631 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 12:17:11,633 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:11,635 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-13 12:17:11,637 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-13 12:17:11,639 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:11,639 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-13 12:17:11,639 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290631639"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:11,639 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290631639"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:11,641 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-13 12:17:11,641 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 223dadc96dff0acc6b1c2fb058f22e1a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676290566318.223dadc96dff0acc6b1c2fb058f22e1a.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 07406c18ee535b7620fbe7fdd2bbe517, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676290566318_0001.07406c18ee535b7620fbe7fdd2bbe517.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-13 12:17:11,641 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-13 12:17:11,641 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676290631641"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:11,642 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-13 12:17:11,648 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:11,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 33 msec 2023-02-13 12:17:11,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 12:17:11,720 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-13 12:17:11,731 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=348 (was 413), OpenFileDescriptor=655 (was 856), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 412), ProcessCount=167 (was 167), AvailableMemoryMB=5451 (was 5210) - AvailableMemoryMB LEAK? - 2023-02-13 12:17:11,743 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=349, OpenFileDescriptor=655, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=167, AvailableMemoryMB=5450 2023-02-13 12:17:11,743 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:17:11,753 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 12:17:11,753 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 12:17:11,755 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:11,756 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:17:11,757 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:17:11,759 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:11,759 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb empty. 2023-02-13 12:17:11,760 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:11,760 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 12:17:11,769 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-13 12:17:11,770 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3fe59b79e126751782de6a355b130ccb, NAME => 'testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:17:11,790 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:11,791 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 3fe59b79e126751782de6a355b130ccb, disabling compactions & flushes 2023-02-13 12:17:11,791 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:11,791 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:11,791 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. after waiting 0 ms 2023-02-13 12:17:11,791 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:11,791 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:11,791 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 3fe59b79e126751782de6a355b130ccb: Waiting for close lock at 1676290631791Disabling compacts and flushes for region at 1676290631791Disabling writes for close at 1676290631791Writing region close event to WAL at 1676290631791Closed at 1676290631791 2023-02-13 12:17:11,793 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:17:11,794 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676290631794"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290631794"}]},"ts":"1676290631794"} 2023-02-13 12:17:11,795 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:17:11,796 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:17:11,796 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290631796"}]},"ts":"1676290631796"} 2023-02-13 12:17:11,798 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-13 12:17:11,814 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:17:11,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:17:11,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:17:11,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:17:11,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:17:11,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=3fe59b79e126751782de6a355b130ccb, ASSIGN}] 2023-02-13 12:17:11,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=3fe59b79e126751782de6a355b130ccb, ASSIGN 2023-02-13 12:17:11,818 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=3fe59b79e126751782de6a355b130ccb, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,45345,1676290533799; forceNewPlan=false, retain=false 2023-02-13 12:17:11,968 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:17:11,969 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=3fe59b79e126751782de6a355b130ccb, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:11,970 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676290631969"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290631969"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290631969"}]},"ts":"1676290631969"} 2023-02-13 12:17:11,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 3fe59b79e126751782de6a355b130ccb, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:17:12,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:12,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3fe59b79e126751782de6a355b130ccb, NAME => 'testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:17:12,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:17:12,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:12,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:12,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:12,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:12,129 INFO [StoreOpener-3fe59b79e126751782de6a355b130ccb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:12,131 DEBUG [StoreOpener-3fe59b79e126751782de6a355b130ccb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1 2023-02-13 12:17:12,131 DEBUG [StoreOpener-3fe59b79e126751782de6a355b130ccb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1 2023-02-13 12:17:12,131 INFO [StoreOpener-3fe59b79e126751782de6a355b130ccb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3fe59b79e126751782de6a355b130ccb columnFamilyName cf1 2023-02-13 12:17:12,132 INFO [StoreOpener-3fe59b79e126751782de6a355b130ccb-1] regionserver.HStore(310): Store=3fe59b79e126751782de6a355b130ccb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:12,132 INFO [StoreOpener-3fe59b79e126751782de6a355b130ccb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:12,134 DEBUG [StoreOpener-3fe59b79e126751782de6a355b130ccb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2 2023-02-13 12:17:12,134 DEBUG [StoreOpener-3fe59b79e126751782de6a355b130ccb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2 2023-02-13 12:17:12,134 INFO [StoreOpener-3fe59b79e126751782de6a355b130ccb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3fe59b79e126751782de6a355b130ccb columnFamilyName cf2 2023-02-13 12:17:12,134 INFO [StoreOpener-3fe59b79e126751782de6a355b130ccb-1] regionserver.HStore(310): Store=3fe59b79e126751782de6a355b130ccb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:12,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:12,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:12,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:17:12,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:12,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:17:12,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 3fe59b79e126751782de6a355b130ccb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74416846, jitterRate=0.10889741778373718}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:17:12,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 3fe59b79e126751782de6a355b130ccb: Running coprocessor pre-open hook at 1676290632127Writing region info on filesystem at 1676290632127Initializing all the Stores at 1676290632128 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290632128Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290632129 (+1 ms)Cleaning up temporary data from old regions at 1676290632136 (+7 ms)Cleaning up detritus from prior splits at 1676290632136Running coprocessor post-open hooks at 1676290632139 (+3 ms)Region opened successfully at 1676290632139 2023-02-13 12:17:12,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb., pid=104, masterSystemTime=1676290632123 2023-02-13 12:17:12,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:12,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:12,142 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=3fe59b79e126751782de6a355b130ccb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,142 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676290632141"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290632141"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290632141"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290632141"}]},"ts":"1676290632141"} 2023-02-13 12:17:12,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-13 12:17:12,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 3fe59b79e126751782de6a355b130ccb, server=jenkins-hbase16.apache.org,45345,1676290533799 in 172 msec 2023-02-13 12:17:12,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-13 12:17:12,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=3fe59b79e126751782de6a355b130ccb, ASSIGN in 328 msec 2023-02-13 12:17:12,146 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:17:12,147 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290632146"}]},"ts":"1676290632146"} 2023-02-13 12:17:12,148 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-13 12:17:12,156 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:17:12,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 402 msec 2023-02-13 12:17:12,263 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:12,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 12:17:12,265 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 12:17:12,265 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,265 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 12:17:12,265 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 12:17:12,266 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,266 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,272 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,272 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,272 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,272 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,273 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,273 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:12,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:12,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,273 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:12,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:12,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:12,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:12,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:12,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:12,274 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:12,274 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:12,275 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. started... 2023-02-13 12:17:12,275 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3fe59b79e126751782de6a355b130ccb 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 12:17:12,280 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,285 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/8c636ce316e2487c9325d9916fddf14f 2023-02-13 12:17:12,289 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,289 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,289 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:12,289 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,289 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:12,319 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/de29320335414074a5ab5a23a7839400 2023-02-13 12:17:12,323 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/8c636ce316e2487c9325d9916fddf14f as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/8c636ce316e2487c9325d9916fddf14f 2023-02-13 12:17:12,327 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/8c636ce316e2487c9325d9916fddf14f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:17:12,328 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/de29320335414074a5ab5a23a7839400 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/de29320335414074a5ab5a23a7839400 2023-02-13 12:17:12,332 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/de29320335414074a5ab5a23a7839400, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:17:12,333 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3fe59b79e126751782de6a355b130ccb in 58ms, sequenceid=6, compaction requested=false 2023-02-13 12:17:12,333 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-13 12:17:12,333 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3fe59b79e126751782de6a355b130ccb: 2023-02-13 12:17:12,333 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:12,333 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:12,333 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:12,333 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,333 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:12,333 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:12,339 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,339 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:12,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:12,339 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:12,339 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:12,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:12,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:12,340 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 12:17:12,340 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@132d6f37[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:12,340 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 12:17:12,341 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,347 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,347 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,347 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,347 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,347 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:12,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,348 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:12,348 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:12,348 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:12,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,348 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:12,348 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:12,348 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:12,348 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:12,348 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:12,348 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:12,355 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:12,355 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,355 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:12,364 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:12,364 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,364 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,364 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:12,364 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,364 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:12,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:12,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:12,364 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:12,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:12,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:12,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:12,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:17:12,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 12:17:12,367 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 12:17:12,367 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:12,367 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:12,367 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 12:17:12,367 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:12,378 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,378 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,378 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:12,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:12,378 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,378 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,378 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,378 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,378 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:12,379 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,379 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:12,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,380 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:12,381 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,381 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,389 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,389 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,389 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,389 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 12:17:12,389 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:12,389 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,389 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,389 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,389 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:12,389 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,389 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:12,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,390 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,490 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:12,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:12,501 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:12,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:12,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 12:17:12,503 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 12:17:12,503 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,503 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 12:17:12,503 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 12:17:12,504 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,504 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,561 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,561 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,561 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,561 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,562 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:12,562 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:12,562 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:12,562 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:12,562 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,562 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,563 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:12,563 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:12,563 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. started... 2023-02-13 12:17:12,563 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:12,563 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3fe59b79e126751782de6a355b130ccb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:12,571 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/701de06cd1bd4485a8598be8317e1dd1 2023-02-13 12:17:12,578 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,585 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/af901cc769204b8582eae397aec47874 2023-02-13 12:17:12,586 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,586 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,586 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:12,587 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,587 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:12,589 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/701de06cd1bd4485a8598be8317e1dd1 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/701de06cd1bd4485a8598be8317e1dd1 2023-02-13 12:17:12,593 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/701de06cd1bd4485a8598be8317e1dd1, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:17:12,594 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/af901cc769204b8582eae397aec47874 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/af901cc769204b8582eae397aec47874 2023-02-13 12:17:12,598 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/af901cc769204b8582eae397aec47874, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:17:12,599 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3fe59b79e126751782de6a355b130ccb in 36ms, sequenceid=19, compaction requested=false 2023-02-13 12:17:12,599 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3fe59b79e126751782de6a355b130ccb: 2023-02-13 12:17:12,599 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:12,599 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:12,599 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:12,600 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,600 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:12,600 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:12,605 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,605 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:12,606 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,606 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:12,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 12:17:12,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2825404f[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:12,608 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 12:17:12,608 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,614 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,614 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,614 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,614 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,614 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:12,614 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,614 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:12,614 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:12,614 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:12,614 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:12,614 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:12,614 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:12,614 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:12,614 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:12,622 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:12,622 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,622 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,622 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:12,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:12,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:12,622 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:12,622 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,623 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:12,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:12,630 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:12,631 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,631 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:12,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:12,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:12,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:17:12,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 12:17:12,633 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 12:17:12,634 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:12,634 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:12,634 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 12:17:12,634 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:12,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:12,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:12,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,639 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,640 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:12,640 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:12,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,642 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,642 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,642 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,653 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 12:17:12,653 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:12,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,653 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,654 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:12,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,654 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,653 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,654 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,654 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,654 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:12,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,654 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,654 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,754 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:12,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:12,768 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:12,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:12,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 12:17:12,770 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 12:17:12,770 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,770 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 12:17:12,770 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 12:17:12,770 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,770 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,780 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,780 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,780 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,780 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,781 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,781 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:12,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:12,782 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:12,782 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:12,782 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:12,782 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:12,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:12,782 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,782 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:12,782 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,782 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:12,782 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:12,783 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:12,783 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:12,783 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:12,783 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:12,783 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:12,783 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:12,783 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:12,783 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:12,783 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:12,783 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:12,783 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. started... 2023-02-13 12:17:12,783 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3fe59b79e126751782de6a355b130ccb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:12,789 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,794 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/2c13f868f38f4303bde8f9efcaf987b5 2023-02-13 12:17:12,797 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,797 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,797 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:12,797 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,797 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:12,810 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/14c4db7e9bd84efdb0deeaa6bc9cb0f3 2023-02-13 12:17:12,815 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/2c13f868f38f4303bde8f9efcaf987b5 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/2c13f868f38f4303bde8f9efcaf987b5 2023-02-13 12:17:12,819 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/2c13f868f38f4303bde8f9efcaf987b5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:17:12,820 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/14c4db7e9bd84efdb0deeaa6bc9cb0f3 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/14c4db7e9bd84efdb0deeaa6bc9cb0f3 2023-02-13 12:17:12,824 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/14c4db7e9bd84efdb0deeaa6bc9cb0f3, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:17:12,825 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3fe59b79e126751782de6a355b130ccb in 42ms, sequenceid=32, compaction requested=true 2023-02-13 12:17:12,825 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3fe59b79e126751782de6a355b130ccb: 2023-02-13 12:17:12,826 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:12,826 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:12,826 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:12,826 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,826 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:12,826 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:12,836 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,836 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:12,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:12,837 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:12,837 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:12,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:12,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:12,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 12:17:12,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10a121d5[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:12,838 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 12:17:12,839 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,847 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,847 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,847 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,847 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,848 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:12,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,848 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:12,848 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:12,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,848 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:12,848 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:12,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,848 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:12,848 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:12,848 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:12,848 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:12,848 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:12,855 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:12,855 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,855 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:12,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:12,856 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:12,864 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:12,864 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,864 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:12,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:12,872 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:12,872 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,872 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:12,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:12,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:12,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:17:12,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 12:17:12,875 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 12:17:12,875 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:12,875 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:12,875 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 12:17:12,875 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:12,880 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,880 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,880 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,880 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,880 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:12,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,881 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,881 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,881 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:12,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,882 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,882 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,883 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:12,883 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:12,884 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,912 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,912 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,912 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:12,912 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,912 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,912 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:12,912 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,912 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 12:17:12,912 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:12,912 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,912 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:12,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:12,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,912 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:12,913 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:12,913 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:12,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,913 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:12,913 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:12,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:12,913 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:12,913 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,913 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:12,913 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:12,913 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:12,913 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,013 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:13,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:13,026 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:13,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:13,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 12:17:13,028 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 12:17:13,028 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,028 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 12:17:13,028 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 12:17:13,028 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,028 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,039 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,039 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,039 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,039 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,039 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,039 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:13,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:13,040 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:13,040 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:13,040 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:13,041 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:13,041 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,041 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:13,041 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:13,041 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:13,041 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. started... 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:13,041 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:13,041 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3fe59b79e126751782de6a355b130ccb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:13,047 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,049 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/3b23995f58ef4d70887adab3b445935e 2023-02-13 12:17:13,055 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,055 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,055 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:13,056 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,056 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:13,059 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/2b957e68d23344d4a6abcb011ee7e8bf 2023-02-13 12:17:13,063 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/3b23995f58ef4d70887adab3b445935e as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/3b23995f58ef4d70887adab3b445935e 2023-02-13 12:17:13,066 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/3b23995f58ef4d70887adab3b445935e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:17:13,067 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/2b957e68d23344d4a6abcb011ee7e8bf as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/2b957e68d23344d4a6abcb011ee7e8bf 2023-02-13 12:17:13,070 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/2b957e68d23344d4a6abcb011ee7e8bf, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:17:13,071 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3fe59b79e126751782de6a355b130ccb in 30ms, sequenceid=45, compaction requested=true 2023-02-13 12:17:13,071 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3fe59b79e126751782de6a355b130ccb: 2023-02-13 12:17:13,071 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:13,071 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:13,071 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:13,072 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,072 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:13,072 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:13,080 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,080 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:13,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:13,081 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,081 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:13,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:13,083 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 12:17:13,083 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c08a83f[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:13,083 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 12:17:13,083 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,089 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,089 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,089 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,089 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:13,089 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:13,089 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:13,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,089 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,089 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:13,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,089 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:13,090 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:13,090 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:13,090 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:13,090 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:13,090 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:13,097 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:13,097 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,097 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:13,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:13,097 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:13,105 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:13,106 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,106 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:13,105 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:13,106 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,106 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:13,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:17:13,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 12:17:13,108 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 12:17:13,108 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:13,108 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:13,108 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 12:17:13,108 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:13,114 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,114 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,114 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:13,114 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,114 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,114 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,114 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,114 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,115 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,115 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:13,116 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,116 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,116 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,122 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,122 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,122 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,122 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,122 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,122 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,122 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,122 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 12:17:13,123 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,122 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,122 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,123 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,223 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:13,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:13,235 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:13,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:13,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 12:17:13,237 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 12:17:13,237 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,237 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 12:17:13,237 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 12:17:13,237 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,238 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,247 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,247 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,247 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,247 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,247 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,247 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:13,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:13,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:13,248 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:13,249 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:13,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:13,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:13,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:13,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:13,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:13,249 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:13,249 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. started... 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:13,249 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:13,250 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3fe59b79e126751782de6a355b130ccb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:13,255 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,264 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,264 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,264 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:13,264 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,264 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:13,265 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/1e5745aca8b7402eae8164dba3e6a761 2023-02-13 12:17:13,279 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/a4ec27c373f94feb983ff5394916f7f5 2023-02-13 12:17:13,283 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/1e5745aca8b7402eae8164dba3e6a761 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/1e5745aca8b7402eae8164dba3e6a761 2023-02-13 12:17:13,286 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/1e5745aca8b7402eae8164dba3e6a761, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:17:13,287 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/a4ec27c373f94feb983ff5394916f7f5 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/a4ec27c373f94feb983ff5394916f7f5 2023-02-13 12:17:13,290 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/a4ec27c373f94feb983ff5394916f7f5, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:17:13,291 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3fe59b79e126751782de6a355b130ccb in 42ms, sequenceid=58, compaction requested=true 2023-02-13 12:17:13,291 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3fe59b79e126751782de6a355b130ccb: 2023-02-13 12:17:13,291 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:13,291 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:13,291 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:13,291 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,291 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:13,291 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:13,321 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,321 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:13,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:13,322 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,322 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:13,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:13,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:13,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:13,324 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 12:17:13,324 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74a42a5e[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:13,324 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 12:17:13,324 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,379 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,379 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,379 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,380 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:13,380 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:13,380 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:13,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,380 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,380 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:13,380 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:13,380 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:13,380 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:13,380 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:13,380 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:13,380 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:13,395 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:13,395 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,395 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:13,403 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:13,403 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,403 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,403 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:13,403 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,403 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:13,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:13,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:13,403 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:13,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:13,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:13,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:13,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:17:13,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 12:17:13,407 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 12:17:13,407 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:13,407 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:13,407 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 12:17:13,407 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:13,414 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,414 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,414 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,414 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:13,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:13,414 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,414 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,414 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:13,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,415 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,415 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,415 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:13,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:13,417 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,417 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,417 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,428 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 12:17:13,429 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,428 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,428 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,429 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:13,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,429 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,429 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,429 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:13,429 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,429 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,430 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,430 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,530 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:13,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:13,541 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:13,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:13,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 12:17:13,542 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 12:17:13,543 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,543 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 12:17:13,543 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 12:17:13,543 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,543 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,553 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,553 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,553 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,553 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,554 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,554 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,554 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:13,555 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:13,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:13,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,555 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 12:17:13,555 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:13,555 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:13,555 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:13,555 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,555 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:13,555 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:13,555 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:13,555 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:13,555 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. started... 2023-02-13 12:17:13,556 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3fe59b79e126751782de6a355b130ccb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:13,564 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,564 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/6d4a5e8676a041f0adaecbcd9a1113f0 2023-02-13 12:17:13,572 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,572 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,572 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:13,572 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,572 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:13,574 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/457dcba5ea16465b96e60dcfc1fa6b0c 2023-02-13 12:17:13,582 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/6d4a5e8676a041f0adaecbcd9a1113f0 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/6d4a5e8676a041f0adaecbcd9a1113f0 2023-02-13 12:17:13,586 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/6d4a5e8676a041f0adaecbcd9a1113f0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:17:13,587 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/457dcba5ea16465b96e60dcfc1fa6b0c as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/457dcba5ea16465b96e60dcfc1fa6b0c 2023-02-13 12:17:13,590 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/457dcba5ea16465b96e60dcfc1fa6b0c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:17:13,591 INFO [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3fe59b79e126751782de6a355b130ccb in 35ms, sequenceid=71, compaction requested=true 2023-02-13 12:17:13,591 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3fe59b79e126751782de6a355b130ccb: 2023-02-13 12:17:13,591 DEBUG [rs(jenkins-hbase16.apache.org,45345,1676290533799)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:13,591 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:13,591 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:13,591 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,591 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 12:17:13,591 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 12:17:13,597 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,597 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:13,598 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,598 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:13,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:13,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:13,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:13,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 12:17:13,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76812bbd[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:13,599 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 12:17:13,599 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,605 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,605 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,605 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,606 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,606 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,606 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:13,606 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:13,606 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:13,606 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 12:17:13,606 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:13,606 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:13,606 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 12:17:13,606 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:13,606 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:13,614 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:13,614 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,614 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:13,622 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:13,622 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,622 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:13,622 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,622 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 12:17:13,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:13,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:13,622 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,623 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 12:17:13,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:13,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:13,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:13,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,45345,1676290533799': 2023-02-13 12:17:13,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 12:17:13,625 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 12:17:13,625 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:13,625 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:13,625 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 12:17:13,625 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:13,630 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,630 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:13,630 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,630 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,631 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,631 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,630 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,631 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,631 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,631 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:13,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,633 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:13,633 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 12:17:13,633 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,639 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 12:17:13,639 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:13,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:13,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,639 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:13,639 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:13,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:13,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:13,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:13,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,639 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:13,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,639 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:13,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:13,640 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,640 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,640 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 12:17:13,640 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:13,640 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,640 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,640 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:13,640 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 12:17:13,640 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 12:17:13,740 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:13,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:13,748 INFO [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,749 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/701de06cd1bd4485a8598be8317e1dd1 because midkey is the same as first or last row 2023-02-13 12:17:13,749 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/af901cc769204b8582eae397aec47874 because midkey is the same as first or last row 2023-02-13 12:17:13,749 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.HRegion(2744): Flushing 3fe59b79e126751782de6a355b130ccb 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 12:17:13,758 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/ae0de1468c9747be8a75331e6b89692f 2023-02-13 12:17:13,769 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/48836f74a452448791a6b85bd14b5a0b 2023-02-13 12:17:13,772 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf1/ae0de1468c9747be8a75331e6b89692f as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/ae0de1468c9747be8a75331e6b89692f 2023-02-13 12:17:13,775 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/ae0de1468c9747be8a75331e6b89692f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:17:13,776 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/.tmp/cf2/48836f74a452448791a6b85bd14b5a0b as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/48836f74a452448791a6b85bd14b5a0b 2023-02-13 12:17:13,780 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/48836f74a452448791a6b85bd14b5a0b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:17:13,781 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3fe59b79e126751782de6a355b130ccb in 32ms, sequenceid=82, compaction requested=true 2023-02-13 12:17:13,782 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.HRegion(2445): Flush status journal for 3fe59b79e126751782de6a355b130ccb: 2023-02-13 12:17:13,782 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/701de06cd1bd4485a8598be8317e1dd1 because midkey is the same as first or last row 2023-02-13 12:17:13,782 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45345] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/af901cc769204b8582eae397aec47874 because midkey is the same as first or last row 2023-02-13 12:17:13,782 DEBUG [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,782 DEBUG [Listener at localhost.localdomain/46087] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-13 12:17:13,782 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:17:13,784 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-13 12:17:13,784 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testInvalidSplitKey 2023-02-13 12:17:13,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:13,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 12:17:13,787 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290633787"}]},"ts":"1676290633787"} 2023-02-13 12:17:13,788 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-13 12:17:13,797 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-13 12:17:13,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=3fe59b79e126751782de6a355b130ccb, UNASSIGN}] 2023-02-13 12:17:13,799 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=3fe59b79e126751782de6a355b130ccb, UNASSIGN 2023-02-13 12:17:13,799 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=3fe59b79e126751782de6a355b130ccb, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:13,800 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676290633799"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290633799"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290633799"}]},"ts":"1676290633799"} 2023-02-13 12:17:13,801 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 3fe59b79e126751782de6a355b130ccb, server=jenkins-hbase16.apache.org,45345,1676290533799}] 2023-02-13 12:17:13,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 12:17:13,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:13,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 3fe59b79e126751782de6a355b130ccb, disabling compactions & flushes 2023-02-13 12:17:13,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:13,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:13,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. after waiting 0 ms 2023-02-13 12:17:13,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:13,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 12:17:13,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:13,958 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb. 2023-02-13 12:17:13,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 3fe59b79e126751782de6a355b130ccb: Waiting for close lock at 1676290633953Running coprocessor pre-close hooks at 1676290633953Disabling compacts and flushes for region at 1676290633953Disabling writes for close at 1676290633953Writing region close event to WAL at 1676290633954 (+1 ms)Running coprocessor post-close hooks at 1676290633957 (+3 ms)Closed at 1676290633958 (+1 ms) 2023-02-13 12:17:13,959 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:13,960 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=3fe59b79e126751782de6a355b130ccb, regionState=CLOSED 2023-02-13 12:17:13,960 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676290633959"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290633959"}]},"ts":"1676290633959"} 2023-02-13 12:17:13,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-13 12:17:13,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 3fe59b79e126751782de6a355b130ccb, server=jenkins-hbase16.apache.org,45345,1676290533799 in 160 msec 2023-02-13 12:17:13,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-13 12:17:13,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=3fe59b79e126751782de6a355b130ccb, UNASSIGN in 164 msec 2023-02-13 12:17:13,964 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290633964"}]},"ts":"1676290633964"} 2023-02-13 12:17:13,965 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-13 12:17:13,972 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-13 12:17:13,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 189 msec 2023-02-13 12:17:14,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 12:17:14,090 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-13 12:17:14,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testInvalidSplitKey 2023-02-13 12:17:14,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:14,093 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:14,094 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:14,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 12:17:14,097 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:14,099 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/recovered.edits] 2023-02-13 12:17:14,102 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/1e5745aca8b7402eae8164dba3e6a761 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/1e5745aca8b7402eae8164dba3e6a761 2023-02-13 12:17:14,104 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/2c13f868f38f4303bde8f9efcaf987b5 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/2c13f868f38f4303bde8f9efcaf987b5 2023-02-13 12:17:14,105 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/3b23995f58ef4d70887adab3b445935e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/3b23995f58ef4d70887adab3b445935e 2023-02-13 12:17:14,106 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/6d4a5e8676a041f0adaecbcd9a1113f0 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/6d4a5e8676a041f0adaecbcd9a1113f0 2023-02-13 12:17:14,107 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/701de06cd1bd4485a8598be8317e1dd1 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/701de06cd1bd4485a8598be8317e1dd1 2023-02-13 12:17:14,108 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/8c636ce316e2487c9325d9916fddf14f to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/8c636ce316e2487c9325d9916fddf14f 2023-02-13 12:17:14,109 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/ae0de1468c9747be8a75331e6b89692f to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf1/ae0de1468c9747be8a75331e6b89692f 2023-02-13 12:17:14,111 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/14c4db7e9bd84efdb0deeaa6bc9cb0f3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/14c4db7e9bd84efdb0deeaa6bc9cb0f3 2023-02-13 12:17:14,112 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/2b957e68d23344d4a6abcb011ee7e8bf to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/2b957e68d23344d4a6abcb011ee7e8bf 2023-02-13 12:17:14,113 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/457dcba5ea16465b96e60dcfc1fa6b0c to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/457dcba5ea16465b96e60dcfc1fa6b0c 2023-02-13 12:17:14,114 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/48836f74a452448791a6b85bd14b5a0b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/48836f74a452448791a6b85bd14b5a0b 2023-02-13 12:17:14,115 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/a4ec27c373f94feb983ff5394916f7f5 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/a4ec27c373f94feb983ff5394916f7f5 2023-02-13 12:17:14,116 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/af901cc769204b8582eae397aec47874 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/af901cc769204b8582eae397aec47874 2023-02-13 12:17:14,117 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/de29320335414074a5ab5a23a7839400 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/cf2/de29320335414074a5ab5a23a7839400 2023-02-13 12:17:14,119 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/recovered.edits/85.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb/recovered.edits/85.seqid 2023-02-13 12:17:14,119 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testInvalidSplitKey/3fe59b79e126751782de6a355b130ccb 2023-02-13 12:17:14,119 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 12:17:14,121 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:14,123 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-13 12:17:14,126 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-13 12:17:14,127 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:14,127 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-13 12:17:14,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290634127"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:14,129 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 12:17:14,129 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3fe59b79e126751782de6a355b130ccb, NAME => 'testInvalidSplitKey,,1676290631754.3fe59b79e126751782de6a355b130ccb.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 12:17:14,129 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-13 12:17:14,129 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676290634129"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:14,130 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-13 12:17:14,139 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:14,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 48 msec 2023-02-13 12:17:14,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 12:17:14,196 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-13 12:17:14,208 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=369 (was 349) - Thread LEAK? -, OpenFileDescriptor=747 (was 655) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 229), ProcessCount=167 (was 167), AvailableMemoryMB=5423 (was 5450) 2023-02-13 12:17:14,219 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=369, OpenFileDescriptor=747, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=167, AvailableMemoryMB=5422 2023-02-13 12:17:14,219 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:17:14,230 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 12:17:14,231 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 12:17:14,233 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:14,235 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:17:14,235 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:17:14,237 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:14,238 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528 empty. 2023-02-13 12:17:14,238 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:14,238 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 12:17:14,249 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 12:17:14,250 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0820811d7bb9089a2330649ab2872528, NAME => 'testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:17:14,262 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:14,262 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 0820811d7bb9089a2330649ab2872528, disabling compactions & flushes 2023-02-13 12:17:14,262 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:14,262 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:14,262 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. after waiting 0 ms 2023-02-13 12:17:14,262 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:14,262 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:14,262 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 0820811d7bb9089a2330649ab2872528: Waiting for close lock at 1676290634262Disabling compacts and flushes for region at 1676290634262Disabling writes for close at 1676290634262Writing region close event to WAL at 1676290634262Closed at 1676290634262 2023-02-13 12:17:14,264 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:17:14,265 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290634265"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290634265"}]},"ts":"1676290634265"} 2023-02-13 12:17:14,266 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:17:14,267 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:17:14,267 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290634267"}]},"ts":"1676290634267"} 2023-02-13 12:17:14,268 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 12:17:14,286 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:17:14,287 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:17:14,287 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:17:14,287 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:17:14,287 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:17:14,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, ASSIGN}] 2023-02-13 12:17:14,289 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, ASSIGN 2023-02-13 12:17:14,290 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:17:14,440 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:17:14,441 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=0820811d7bb9089a2330649ab2872528, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,441 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290634441"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290634441"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290634441"}]},"ts":"1676290634441"} 2023-02-13 12:17:14,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:14,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:14,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0820811d7bb9089a2330649ab2872528, NAME => 'testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:17:14,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:17:14,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:14,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:14,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:14,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:14,599 INFO [StoreOpener-0820811d7bb9089a2330649ab2872528-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:14,600 DEBUG [StoreOpener-0820811d7bb9089a2330649ab2872528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1 2023-02-13 12:17:14,600 DEBUG [StoreOpener-0820811d7bb9089a2330649ab2872528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1 2023-02-13 12:17:14,601 INFO [StoreOpener-0820811d7bb9089a2330649ab2872528-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0820811d7bb9089a2330649ab2872528 columnFamilyName cf1 2023-02-13 12:17:14,601 INFO [StoreOpener-0820811d7bb9089a2330649ab2872528-1] regionserver.HStore(310): Store=0820811d7bb9089a2330649ab2872528/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:14,601 INFO [StoreOpener-0820811d7bb9089a2330649ab2872528-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:14,602 DEBUG [StoreOpener-0820811d7bb9089a2330649ab2872528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2 2023-02-13 12:17:14,602 DEBUG [StoreOpener-0820811d7bb9089a2330649ab2872528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2 2023-02-13 12:17:14,603 INFO [StoreOpener-0820811d7bb9089a2330649ab2872528-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0820811d7bb9089a2330649ab2872528 columnFamilyName cf2 2023-02-13 12:17:14,603 INFO [StoreOpener-0820811d7bb9089a2330649ab2872528-1] regionserver.HStore(310): Store=0820811d7bb9089a2330649ab2872528/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:14,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:14,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:14,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:17:14,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:14,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:17:14,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 0820811d7bb9089a2330649ab2872528; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70783658, jitterRate=0.054758697748184204}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:17:14,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 0820811d7bb9089a2330649ab2872528: Running coprocessor pre-open hook at 1676290634598Writing region info on filesystem at 1676290634598Initializing all the Stores at 1676290634599 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290634599Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290634599Cleaning up temporary data from old regions at 1676290634605 (+6 ms)Cleaning up detritus from prior splits at 1676290634605Running coprocessor post-open hooks at 1676290634608 (+3 ms)Region opened successfully at 1676290634608 2023-02-13 12:17:14,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528., pid=111, masterSystemTime=1676290634594 2023-02-13 12:17:14,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:14,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:14,611 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=0820811d7bb9089a2330649ab2872528, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,611 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290634611"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290634611"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290634611"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290634611"}]},"ts":"1676290634611"} 2023-02-13 12:17:14,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-13 12:17:14,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 in 169 msec 2023-02-13 12:17:14,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-13 12:17:14,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, ASSIGN in 325 msec 2023-02-13 12:17:14,615 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:17:14,616 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290634616"}]},"ts":"1676290634616"} 2023-02-13 12:17:14,617 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 12:17:14,673 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:17:14,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 441 msec 2023-02-13 12:17:14,741 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:14,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 12:17:14,742 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:14,743 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:14,743 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 12:17:14,743 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:17:14,743 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,743 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,753 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:14,753 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:14,753 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:14,753 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:14,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:14,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:14,754 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,754 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:14,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:14,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,754 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:14,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:14,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:14,755 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:14,755 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:14,755 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:14,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:14,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:14,755 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:14,755 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:14,755 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:14,755 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:14,755 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:14,755 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. started... 2023-02-13 12:17:14,755 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:14,755 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:14,757 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0820811d7bb9089a2330649ab2872528 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 12:17:14,757 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:14,755 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:14,757 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:14,757 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:14,757 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:14,757 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:14,764 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,764 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,764 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,764 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:14,764 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,764 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:14,767 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/21932edcea39499089bb0178bc96fcf2 2023-02-13 12:17:14,781 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/4738c514b09f438187452c4befd8d3df 2023-02-13 12:17:14,784 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/21932edcea39499089bb0178bc96fcf2 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2 2023-02-13 12:17:14,788 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:17:14,789 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/4738c514b09f438187452c4befd8d3df as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df 2023-02-13 12:17:14,793 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:17:14,794 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 0820811d7bb9089a2330649ab2872528 in 37ms, sequenceid=6, compaction requested=false 2023-02-13 12:17:14,794 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:14,794 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0820811d7bb9089a2330649ab2872528: 2023-02-13 12:17:14,794 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:14,794 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:14,794 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:14,794 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,794 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:14,794 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:14,803 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,803 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:14,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:14,803 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,803 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:14,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:14,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:14,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:14,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:14,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:14,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:14,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 12:17:14,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f371331[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:14,805 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:17:14,805 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,814 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,814 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,814 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,814 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,814 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:14,814 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:14,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,814 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:14,814 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:14,814 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:14,815 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:14,815 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:14,814 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:14,815 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:14,815 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:14,822 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:14,822 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:14,822 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:14,830 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:14,830 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,830 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:14,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,830 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:14,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:14,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:14,831 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:14,831 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:14,831 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:14,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:14,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:14,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:14,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:14,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:14,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:14,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:14,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:14,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:14,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:14,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:17:14,834 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 12:17:14,834 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:14,834 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:14,834 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 12:17:14,834 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:14,845 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:14,845 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:14,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:14,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:14,846 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:14,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:14,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:14,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:14,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,846 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:14,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:14,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:14,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:14,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:14,847 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:14,847 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:14,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:14,848 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:14,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:14,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:14,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:14,855 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:14,855 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:14,855 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:14,855 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:14,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:14,856 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:14,856 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:14,856 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:14,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:14,856 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:14,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:14,856 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:14,856 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:14,856 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:14,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:14,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:14,856 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:14,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:14,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:14,856 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:14,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:14,856 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:14,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:14,856 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:14,856 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:14,857 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:14,857 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,857 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:14,857 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,857 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,857 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,957 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:14,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:14,967 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:14,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:14,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 12:17:14,969 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:14,969 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:14,969 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 12:17:14,969 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:17:14,969 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,969 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,978 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,978 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:14,978 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:14,978 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:14,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:14,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:14,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:14,978 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:14,978 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:14,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:14,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:14,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:14,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:14,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:14,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:14,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:14,980 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:14,980 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. started... 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:14,980 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:14,980 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0820811d7bb9089a2330649ab2872528 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:14,988 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/a267453befc043d18fb0816dd2c95854 2023-02-13 12:17:14,998 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/2b3c57f7e47b4fcd94e2b424dad0c387 2023-02-13 12:17:15,001 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/a267453befc043d18fb0816dd2c95854 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854 2023-02-13 12:17:15,005 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:17:15,005 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/2b3c57f7e47b4fcd94e2b424dad0c387 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387 2023-02-13 12:17:15,009 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:17:15,010 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0820811d7bb9089a2330649ab2872528 in 29ms, sequenceid=19, compaction requested=false 2023-02-13 12:17:15,010 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0820811d7bb9089a2330649ab2872528: 2023-02-13 12:17:15,010 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:15,010 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:15,010 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:15,010 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,010 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,010 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,021 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,039 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,039 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,039 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,047 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,047 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,047 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,047 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,047 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,047 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 12:17:15,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@263717b8[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:15,049 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:17:15,049 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,055 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,055 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,055 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,056 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,056 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:15,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,056 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,056 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,056 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,056 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:15,056 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,056 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,056 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,056 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:15,056 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:15,064 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,064 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,064 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,064 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,072 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,072 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,072 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,072 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,072 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,073 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:15,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:17:15,075 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 12:17:15,075 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:15,075 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:15,075 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,075 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:15,080 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,080 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,080 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,080 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,081 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,081 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,081 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,081 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,082 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,082 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,083 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,083 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,083 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,095 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,095 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:15,095 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:15,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:15,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,095 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:15,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,096 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,096 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,096 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,096 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,096 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,096 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,096 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,096 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,196 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:15,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:15,207 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:15,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:15,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 12:17:15,209 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:15,209 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,209 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 12:17:15,209 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:17:15,210 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,210 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,220 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,220 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,220 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,220 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,220 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,220 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:15,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,221 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,221 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,221 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,222 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,222 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,222 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,222 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. started... 2023-02-13 12:17:15,222 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0820811d7bb9089a2330649ab2872528 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:15,229 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/df9cb29ea40640b794d94d4e7c7da680 2023-02-13 12:17:15,230 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,230 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,231 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,231 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,231 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,231 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,241 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/c58bf212ff804449a1fded2bbeebed5e 2023-02-13 12:17:15,245 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/df9cb29ea40640b794d94d4e7c7da680 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680 2023-02-13 12:17:15,249 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:17:15,249 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/c58bf212ff804449a1fded2bbeebed5e as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e 2023-02-13 12:17:15,253 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:17:15,254 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0820811d7bb9089a2330649ab2872528 in 32ms, sequenceid=32, compaction requested=true 2023-02-13 12:17:15,254 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0820811d7bb9089a2330649ab2872528: 2023-02-13 12:17:15,254 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:15,254 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:15,254 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:15,254 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,254 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,254 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,264 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,264 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,264 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,264 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 12:17:15,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73142ffa[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:15,266 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:17:15,266 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,272 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,272 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,272 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,273 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,273 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:15,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,273 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,273 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,273 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,273 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:15,273 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,273 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:15,273 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,273 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,273 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:15,281 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,281 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,281 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,315 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,315 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,315 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,315 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,315 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,315 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,316 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:15,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:17:15,319 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 12:17:15,319 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:15,319 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:15,319 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,319 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:15,331 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,331 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,331 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,331 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,331 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,331 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,331 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,331 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,331 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,332 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,333 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,333 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,333 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,345 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,345 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,345 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,345 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,345 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,345 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,345 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,345 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,345 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,345 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,345 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:15,345 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:15,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,345 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,346 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,346 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,346 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,346 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,346 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,346 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,346 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:15,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,346 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,346 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:15,446 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:15,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:15,458 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:15,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:15,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 12:17:15,459 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:15,459 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,459 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 12:17:15,459 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:17:15,459 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,460 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,470 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,470 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,470 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,470 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,470 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,471 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:15,472 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:15,472 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:15,472 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. started... 2023-02-13 12:17:15,472 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0820811d7bb9089a2330649ab2872528 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:15,481 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,483 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/f862226517df4fd98f78ba440e75881c 2023-02-13 12:17:15,489 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,489 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,489 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,489 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,489 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,493 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/b70e7918f9e9496c863a4b2dc588b4a5 2023-02-13 12:17:15,497 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/f862226517df4fd98f78ba440e75881c as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c 2023-02-13 12:17:15,500 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:17:15,501 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/b70e7918f9e9496c863a4b2dc588b4a5 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5 2023-02-13 12:17:15,505 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:17:15,508 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0820811d7bb9089a2330649ab2872528 in 36ms, sequenceid=45, compaction requested=true 2023-02-13 12:17:15,508 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0820811d7bb9089a2330649ab2872528: 2023-02-13 12:17:15,508 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:15,508 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:15,508 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:15,508 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,508 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,508 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,514 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,514 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,514 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,514 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 12:17:15,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c373ee6[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:15,516 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:17:15,516 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,522 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,522 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,522 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,522 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,523 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:15,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,523 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,523 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,523 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:15,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,523 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,523 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,523 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,523 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,523 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:15,523 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:15,531 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,531 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,531 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,539 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,539 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,539 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,539 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,539 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,539 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,539 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:15,541 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:17:15,541 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 12:17:15,542 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:15,542 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:15,542 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,542 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:15,547 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,547 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,547 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,547 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,547 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,547 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,547 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,548 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,548 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,548 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,550 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,550 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,550 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,561 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,561 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,562 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,562 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,561 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,562 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:15,562 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:15,562 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,562 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,562 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,562 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,562 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:15,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,562 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,563 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:15,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,563 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,563 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,563 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,563 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,563 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,563 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,563 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,563 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,663 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:15,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:15,674 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:15,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:15,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 12:17:15,676 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:15,676 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,676 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 12:17:15,676 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:17:15,677 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,677 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,686 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,686 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,686 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,686 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,687 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,687 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:15,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,688 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,688 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,688 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,688 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:15,688 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:15,688 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:15,688 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. started... 2023-02-13 12:17:15,689 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0820811d7bb9089a2330649ab2872528 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:15,696 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/5e7fe42c78644940a6be41ff9c2ebdc0 2023-02-13 12:17:15,709 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/7f1af891ca3a444592fcac07a55d29a3 2023-02-13 12:17:15,713 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/5e7fe42c78644940a6be41ff9c2ebdc0 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0 2023-02-13 12:17:15,716 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,717 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:17:15,718 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/7f1af891ca3a444592fcac07a55d29a3 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3 2023-02-13 12:17:15,721 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:17:15,722 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0820811d7bb9089a2330649ab2872528 in 33ms, sequenceid=58, compaction requested=true 2023-02-13 12:17:15,722 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0820811d7bb9089a2330649ab2872528: 2023-02-13 12:17:15,722 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:15,722 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:15,722 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:15,722 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,722 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,722 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,728 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,728 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,728 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,736 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,736 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,736 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,737 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,737 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,737 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,738 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 12:17:15,738 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5bce1aca[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:15,738 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:17:15,739 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,747 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,747 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,748 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,748 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:15,747 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,748 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:15,748 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,748 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,748 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,748 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:15,748 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:15,755 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,756 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,756 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,755 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,764 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,764 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,764 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,764 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,764 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,764 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:15,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:17:15,767 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 12:17:15,767 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:15,767 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:15,767 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,767 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:15,772 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,772 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,772 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,772 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,772 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,772 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,773 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,773 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,773 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,773 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,775 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,775 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,775 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,786 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,786 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,787 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,787 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,787 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,786 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:15,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:15,786 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,787 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,787 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,787 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,787 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,786 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,787 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,787 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,787 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:15,787 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:15,788 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:15,788 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,788 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,788 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,888 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:15,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:15,899 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:15,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:15,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 12:17:15,901 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:15,901 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,901 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 12:17:15,901 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 12:17:15,901 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,902 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,911 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,911 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,911 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,911 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:15,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:15,912 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,912 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:15,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:15,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 12:17:15,913 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,913 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,913 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,913 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 12:17:15,913 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,914 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:15,914 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:15,914 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,914 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:15,914 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,914 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. started... 2023-02-13 12:17:15,914 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0820811d7bb9089a2330649ab2872528 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:15,921 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/0c08830661364a319f483baf2fbce4a9 2023-02-13 12:17:15,922 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,930 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,931 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,931 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,931 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,931 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,932 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd 2023-02-13 12:17:15,936 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/0c08830661364a319f483baf2fbce4a9 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9 2023-02-13 12:17:15,940 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:17:15,941 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd 2023-02-13 12:17:15,945 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:17:15,946 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0820811d7bb9089a2330649ab2872528 in 32ms, sequenceid=71, compaction requested=true 2023-02-13 12:17:15,946 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0820811d7bb9089a2330649ab2872528: 2023-02-13 12:17:15,946 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:15,946 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:15,946 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:15,946 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,946 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 12:17:15,946 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 12:17:15,955 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,955 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:15,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:15,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:15,956 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,956 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:15,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:15,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:15,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:15,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:15,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:15,957 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 12:17:15,957 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@455a5cdc[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:15,957 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 12:17:15,957 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,964 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:15,964 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:15,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,964 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,964 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:15,964 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:15,964 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,964 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 12:17:15,965 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,965 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:15,965 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 12:17:15,965 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:15,972 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:15,972 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:15,972 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:16,000 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:16,001 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:16,001 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:16,000 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,000 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 12:17:16,001 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:16,001 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 12:17:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:16,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:16,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:16,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:16,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:16,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:16,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:16,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 12:17:16,004 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 12:17:16,004 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:16,004 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:16,004 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 12:17:16,004 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:16,014 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,014 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,014 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,014 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,014 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:16,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,014 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:16,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:16,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:16,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,014 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:16,014 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:16,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:16,015 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:16,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:16,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:16,015 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:16,016 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:16,017 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 12:17:16,017 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:16,028 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:16,028 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:16,028 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:16,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:16,028 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:16,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:16,028 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:16,028 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:16,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:16,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:16,028 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:16,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:16,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:16,028 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 12:17:16,029 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:16,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:16,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:16,028 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:16,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:16,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:16,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:16,029 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:16,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:16,029 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,029 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:16,029 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:16,029 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,029 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,029 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:16,029 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,029 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:16,029 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,030 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,030 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 12:17:16,129 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:16,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:16,388 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 12:17:16,389 DEBUG [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:16,390 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,391 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,391 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,391 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,392 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:16,392 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:16,392 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:16,640 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,640 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:16,641 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:16,641 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:16,641 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-13 12:17:16,641 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:16,641 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:16,641 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:16,641 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:16,642 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:16,643 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 12:17:16,644 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:16,644 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:16,644 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:16,644 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:16,644 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:16,645 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:16,645 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:16,645 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:16,645 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:16,645 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:16,645 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:16,645 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:16,646 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:16,646 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:16,646 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:16,646 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:16,646 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:16,646 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:16,646 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:16,647 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:16,647 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:16,647 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:16,647 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:16,647 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:16,647 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:16,647 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:16,648 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:16,648 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:16,648 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:16,648 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:16,648 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:16,648 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:16,649 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:16,649 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:16,649 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,649 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:16,649 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,649 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,649 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,649 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:17:16,650 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:16,651 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:16,651 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290666651 2023-02-13 12:17:16,652 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,652 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:16,655 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:16,655 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:16,655 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:16,655 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290696655 2023-02-13 12:17:16,655 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290756655 2023-02-13 12:17:16,655 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 12:17:16,655 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:16,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:16,657 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:16,658 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:16,729 ERROR [BP-591480327-188.40.62.62-1676290528441 heartbeating to localhost.localdomain/127.0.0.1:44273] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-591480327-188.40.62.62-1676290528441 (Datanode Uuid 14b0d84a-bcac-40f1-8e55-f251954dc488) service to localhost.localdomain/127.0.0.1:44273 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 12:17:16,905 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,905 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:16,905 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:16,905 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:16,906 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:17:16,906 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:16,906 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:16,906 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:16,906 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:16,907 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:16,908 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 12:17:16,908 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:16,908 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:16,908 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:16,908 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:16,909 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:16,909 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:16,909 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:16,909 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:16,909 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:16,909 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:16,909 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:16,909 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:16,910 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:16,910 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:16,910 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:16,910 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:16,910 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:16,910 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:16,910 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:16,911 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:16,911 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:16,911 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:16,911 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:16,911 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:16,911 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:16,911 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:16,911 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:16,911 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:16,912 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:16,912 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:16,912 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:16,912 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:16,912 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:16,912 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:16,912 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,912 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:16,913 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,913 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,913 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,913 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 12:17:16,914 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:16,914 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:16,914 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290666914 2023-02-13 12:17:16,915 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,915 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:16,918 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:16,918 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:16,918 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:16,918 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290696918 2023-02-13 12:17:16,918 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290756918 2023-02-13 12:17:16,918 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 12:17:16,919 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:16,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN}] 2023-02-13 12:17:16,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:16,921 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:16,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:16,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:17,049 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:17:17,169 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,169 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:17,169 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:17,169 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:17,169 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:17:17,169 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:17,169 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:17,169 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:17,169 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:17,170 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:17,171 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 12:17:17,171 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:17,172 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:17,172 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:17,172 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:17,172 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:17,172 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:17,173 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:17,173 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:17,173 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:17,173 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:17,173 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:17,173 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:17,173 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:17,174 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:17,174 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:17,174 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:17,174 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:17,175 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:17,175 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:17,175 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:17,175 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:17,175 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:17,176 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:17,176 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:17,176 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:17,176 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:17,177 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:17,177 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:17,177 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:17,177 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:17,178 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:17,178 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:17,178 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:17,178 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:17,178 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,179 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:17,179 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,179 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,179 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,179 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 12:17:17,180 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:17,180 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:17,181 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290667181 2023-02-13 12:17:17,182 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,182 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:17,185 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:17,185 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:17,185 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:17,185 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290697185 2023-02-13 12:17:17,186 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290757186 2023-02-13 12:17:17,186 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:17:17,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN}] 2023-02-13 12:17:17,186 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:17,188 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN 2023-02-13 12:17:17,189 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=0820811d7bb9089a2330649ab2872528, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:17,189 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290637189"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290637189"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290637189"}]},"ts":"1676290637189"} 2023-02-13 12:17:17,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:17,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 0820811d7bb9089a2330649ab2872528, disabling compactions & flushes 2023-02-13 12:17:17,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:17,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:17,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. after waiting 0 ms 2023-02-13 12:17:17,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:17,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 0820811d7bb9089a2330649ab2872528 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 12:17:17,352 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/1c5d02981368482ba2d2d33ba702a26e 2023-02-13 12:17:17,362 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/05b6753f918044d1833b071aabfddeb2 2023-02-13 12:17:17,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf1/1c5d02981368482ba2d2d33ba702a26e as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e 2023-02-13 12:17:17,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:17:17,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/.tmp/cf2/05b6753f918044d1833b071aabfddeb2 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2 2023-02-13 12:17:17,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:17:17,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 0820811d7bb9089a2330649ab2872528 in 31ms, sequenceid=82, compaction requested=true 2023-02-13 12:17:17,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 12:17:17,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:17,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. 2023-02-13 12:17:17,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 0820811d7bb9089a2330649ab2872528: Waiting for close lock at 1676290637342Running coprocessor pre-close hooks at 1676290637342Disabling compacts and flushes for region at 1676290637342Disabling writes for close at 1676290637342Obtaining lock to block concurrent updates at 1676290637343 (+1 ms)Preparing flush snapshotting stores in 0820811d7bb9089a2330649ab2872528 at 1676290637343Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676290637343Flushing stores of testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. at 1676290637343Flushing 0820811d7bb9089a2330649ab2872528/cf1: creating writer at 1676290637344 (+1 ms)Flushing 0820811d7bb9089a2330649ab2872528/cf1: appending metadata at 1676290637346 (+2 ms)Flushing 0820811d7bb9089a2330649ab2872528/cf1: closing flushed file at 1676290637346Flushing 0820811d7bb9089a2330649ab2872528/cf2: creating writer at 1676290637355 (+9 ms)Flushing 0820811d7bb9089a2330649ab2872528/cf2: appending metadata at 1676290637357 (+2 ms)Flushing 0820811d7bb9089a2330649ab2872528/cf2: closing flushed file at 1676290637357Flushing 0820811d7bb9089a2330649ab2872528/cf1: reopening flushed file at 1676290637366 (+9 ms)Flushing 0820811d7bb9089a2330649ab2872528/cf2: reopening flushed file at 1676290637371 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 0820811d7bb9089a2330649ab2872528 in 31ms, sequenceid=82, compaction requested=true at 1676290637374 (+3 ms)Writing region close event to WAL at 1676290637375 (+1 ms)Running coprocessor post-close hooks at 1676290637378 (+3 ms)Closed at 1676290637379 (+1 ms) 2023-02-13 12:17:17,381 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,382 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=0820811d7bb9089a2330649ab2872528, regionState=CLOSED 2023-02-13 12:17:17,382 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290637382"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290637382"}]},"ts":"1676290637382"} 2023-02-13 12:17:17,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-13 12:17:17,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 in 194 msec 2023-02-13 12:17:17,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-13 12:17:17,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN in 200 msec 2023-02-13 12:17:17,389 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:17,389 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:17,389 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:17,436 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,436 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:17,436 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:17,436 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:17,436 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:17:17,437 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:17,437 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:17,437 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:17,437 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:17,437 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:17,439 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 12:17:17,439 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:17:17,439 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:17:17,440 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:17,440 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:17,440 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:17,440 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:17,441 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:17,441 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:17,441 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:17,441 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:17,441 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:17,442 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:17,442 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:17,442 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:17,442 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:17,442 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:17,443 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:17,443 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:17,443 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:17,443 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:17,443 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:17,443 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:17,444 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:17,444 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:17,444 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:17,444 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:17,444 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:17,445 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:17,445 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:17,445 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:17,446 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:17,446 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:17,446 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:17,446 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:17,446 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:17,446 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:17,447 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,447 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN 2023-02-13 12:17:17,447 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:17,447 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:17,447 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,447 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,447 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,447 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:17,447 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:17,447 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 12:17:17,448 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:17,449 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:17,449 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290667449 2023-02-13 12:17:17,450 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:17,450 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:17,453 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:17,453 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:17,453 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:17,453 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290697453 2023-02-13 12:17:17,453 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290757453 2023-02-13 12:17:17,453 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:17:17,456 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:17,466 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=0820811d7bb9089a2330649ab2872528, threads=14 2023-02-13 12:17:17,466 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,466 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,467 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,617 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,633 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,642 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,648 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,666 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,678 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,679 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,680 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,680 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,681 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,681 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,681 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:17,681 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,067 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,067 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 0820811d7bb9089a2330649ab2872528 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 12:17:18,102 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:18,102 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,102 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:18,102 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:18,204 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,204 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:18,204 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:18,204 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:18,204 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:17:18,204 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:18,204 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:18,205 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:18,205 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:18,205 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:18,206 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 12:17:18,207 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:17:18,207 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:17:18,207 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:18,207 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:18,207 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:18,207 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:18,208 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:18,208 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:18,208 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:18,208 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:18,208 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:18,208 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:18,208 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:18,208 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:18,209 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:18,209 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:18,209 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:18,209 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:18,209 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:18,209 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:18,209 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:18,210 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:18,210 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:18,210 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:18,210 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:18,210 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:18,210 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:18,210 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,210 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,211 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:18,211 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,211 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,211 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:18,211 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:18,211 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:18,211 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:18,211 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,212 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN 2023-02-13 12:17:18,212 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:18,212 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:18,212 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,212 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,212 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,212 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:18,212 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:18,212 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 12:17:18,213 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:18,213 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:18,214 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290668214 2023-02-13 12:17:18,215 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:18,215 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,217 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:18,218 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:18,218 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:18,218 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290698218 2023-02-13 12:17:18,218 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290758218 2023-02-13 12:17:18,218 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:17:18,226 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=0820811d7bb9089a2330649ab2872528, threads=14 2023-02-13 12:17:18,226 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,226 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,226 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,226 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,226 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,226 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,226 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,227 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,227 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,227 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,227 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,227 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,227 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,227 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,360 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,360 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,369 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,384 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,385 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,396 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,397 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,408 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,408 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,408 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,412 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,412 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387 for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,414 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,414 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df for region: 0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:18,414 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 0820811d7bb9089a2330649ab2872528 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 12:17:18,445 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:18,449 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:17:18,451 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:17:18,451 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:18,451 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:18,451 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:18,468 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,468 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:18,468 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:18,469 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:18,469 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:17:18,469 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:18,469 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:18,469 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:18,469 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:18,470 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:18,471 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 12:17:18,471 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:17:18,471 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:17:18,471 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:18,472 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:18,472 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:18,472 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:18,472 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:18,472 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:18,473 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:18,473 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:18,473 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:18,473 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:18,473 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:18,473 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:18,473 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:18,474 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:18,474 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:18,474 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:18,474 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:18,474 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:18,474 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:18,474 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:18,474 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:18,475 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:18,475 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:18,475 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:18,475 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:18,475 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,475 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,476 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:18,476 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,476 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,476 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:18,476 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:18,476 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:18,476 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:18,477 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,477 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN 2023-02-13 12:17:18,477 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:18,477 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:18,477 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,477 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,477 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,477 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:18,477 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:18,477 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:17:18,478 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:18,478 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:18,479 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290668479 2023-02-13 12:17:18,480 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,480 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:18,482 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:18,482 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:18,483 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:18,483 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290698483 2023-02-13 12:17:18,483 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290758483 2023-02-13 12:17:18,483 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 12:17:18,485 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:18,487 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:18,487 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:18,487 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:18,733 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,733 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:18,733 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:18,733 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:18,733 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:17:18,734 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:18,734 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:18,734 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:18,734 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:18,734 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:18,735 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 12:17:18,735 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:17:18,735 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:17:18,735 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:18,736 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:18,736 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:18,736 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:18,736 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:18,736 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:18,736 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:18,736 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:18,736 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:18,737 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:18,737 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:18,737 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:18,737 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:18,737 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:18,737 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:18,737 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:18,737 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:18,737 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:18,738 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:18,738 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:18,738 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:18,738 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:18,738 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:18,738 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:18,738 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:18,738 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,738 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,739 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:18,739 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,739 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:18,739 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:18,739 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:18,739 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:18,739 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:18,740 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,740 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN 2023-02-13 12:17:18,740 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:18,740 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:18,740 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,740 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,740 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,740 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:18,741 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:18,741 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 12:17:18,741 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:18,742 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:18,742 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290668742 2023-02-13 12:17:18,744 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,744 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:18,746 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:18,746 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:18,747 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:18,747 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290698747 2023-02-13 12:17:18,747 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290758747 2023-02-13 12:17:18,747 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 12:17:18,747 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:18,750 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290638749"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676290638749"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676290638749"}]},"ts":"1676290638749"} 2023-02-13 12:17:18,750 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290638749"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290638749"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290638749"}]},"ts":"1676290638749"} 2023-02-13 12:17:18,750 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290638749"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290638749"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290638749"}]},"ts":"1676290638749"} 2023-02-13 12:17:18,752 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:18,752 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:18,752 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:18,997 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:18,997 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:18,997 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:18,997 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:18,997 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:17:18,998 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:18,998 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:18,998 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:18,998 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:18,998 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:18,999 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 12:17:18,999 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:17:19,000 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:17:19,000 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:19,000 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:19,001 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:19,001 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:19,001 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:19,001 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:19,001 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:19,001 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:19,002 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:19,002 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:19,002 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:19,002 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:19,002 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:19,003 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:19,003 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:19,003 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:19,003 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:19,004 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:19,004 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:19,004 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:19,004 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:19,004 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:19,004 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:19,004 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:19,004 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:19,005 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,005 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,005 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:19,005 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,005 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,006 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:19,006 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:19,006 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:19,006 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:19,006 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,006 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN 2023-02-13 12:17:19,006 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:19,006 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:19,006 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,006 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,006 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,007 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:19,007 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:19,007 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 12:17:19,007 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:19,008 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:19,009 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,009 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290669008 2023-02-13 12:17:19,010 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:19,012 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:19,013 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:19,013 INFO [Listener at localhost.localdomain/46087] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:19,013 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=f4e141a29d09bf2245d3b36131601a88, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 12:17:19,014 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=39abe5dfce1992816479263273d22d55, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 12:17:19,014 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:19,014 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290699014 2023-02-13 12:17:19,014 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290759014 2023-02-13 12:17:19,014 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 12:17:19,015 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676290639014"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676290639014"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676290639014"}]},"ts":"1676290639014"} 2023-02-13 12:17:19,015 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290639014"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290639014"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290639014"}]},"ts":"1676290639014"} 2023-02-13 12:17:19,015 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290639014"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290639014"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290639014"}]},"ts":"1676290639014"} 2023-02-13 12:17:19,016 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:19,019 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:19,019 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,019 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:19,019 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:19,264 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,265 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:19,265 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:19,265 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:19,265 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:17:19,265 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:19,265 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:19,265 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:19,265 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:19,266 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:19,267 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-13 12:17:19,267 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:17:19,267 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:17:19,267 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:19,267 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:19,268 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:19,268 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:19,268 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:19,268 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:19,268 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:19,268 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:19,269 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:19,269 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:19,269 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:19,269 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:19,269 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:19,270 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:19,270 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:19,270 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:19,270 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:19,270 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:19,270 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:19,270 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:19,271 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:19,271 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:19,271 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:19,271 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:19,271 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:19,271 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,271 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,272 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:19,272 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,272 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,272 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:19,272 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:19,272 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:19,273 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:19,273 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,273 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN 2023-02-13 12:17:19,273 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:19,273 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:19,273 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,273 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,273 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,273 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:19,274 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:19,274 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 12:17:19,274 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:19,275 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:19,275 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290669275 2023-02-13 12:17:19,276 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:19,277 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,279 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:19,279 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:19,279 INFO [Listener at localhost.localdomain/46087] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:19,279 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=f4e141a29d09bf2245d3b36131601a88, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 12:17:19,280 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=39abe5dfce1992816479263273d22d55, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 12:17:19,280 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:19,280 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290699280 2023-02-13 12:17:19,280 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290759280 2023-02-13 12:17:19,280 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 12:17:19,280 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:19,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f4e141a29d09bf2245d3b36131601a88, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=39abe5dfce1992816479263273d22d55, ASSIGN}] 2023-02-13 12:17:19,283 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:19,283 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:19,283 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:19,530 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,530 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:19,530 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:19,531 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:19,531 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:17:19,531 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:19,531 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:19,531 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:19,531 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:19,532 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:19,533 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-13 12:17:19,533 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:17:19,533 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:17:19,533 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:19,533 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:19,533 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:19,533 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:19,534 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:19,534 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:19,534 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:19,534 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:19,534 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:19,534 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:19,534 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:19,534 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:19,535 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:19,535 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:19,535 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:19,535 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:19,535 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:19,535 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:19,535 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:19,535 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:19,536 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:19,536 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:19,536 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:19,536 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:19,536 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:19,536 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,536 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,537 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:19,537 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,537 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:19,537 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:19,537 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:19,537 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:19,537 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:19,538 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,538 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN 2023-02-13 12:17:19,538 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:19,538 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:19,538 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,538 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,538 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,538 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:19,538 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:19,538 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 12:17:19,539 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:19,539 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:19,540 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290669540 2023-02-13 12:17:19,540 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:19,541 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,543 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:19,543 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:19,543 INFO [Listener at localhost.localdomain/46087] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:19,544 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=f4e141a29d09bf2245d3b36131601a88, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 12:17:19,544 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=39abe5dfce1992816479263273d22d55, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 12:17:19,544 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:19,544 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290699544 2023-02-13 12:17:19,544 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290759544 2023-02-13 12:17:19,544 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 12:17:19,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f4e141a29d09bf2245d3b36131601a88, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=39abe5dfce1992816479263273d22d55, ASSIGN}] 2023-02-13 12:17:19,544 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:19,546 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=39abe5dfce1992816479263273d22d55, ASSIGN 2023-02-13 12:17:19,547 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=39abe5dfce1992816479263273d22d55, ASSIGN; state=CLOSED, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:17:19,547 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f4e141a29d09bf2245d3b36131601a88, ASSIGN 2023-02-13 12:17:19,548 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f4e141a29d09bf2245d3b36131601a88, ASSIGN; state=CLOSED, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:17:19,697 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 12:17:19,698 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=f4e141a29d09bf2245d3b36131601a88, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:19,699 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290639698"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290639698"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290639698"}]},"ts":"1676290639698"} 2023-02-13 12:17:19,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure f4e141a29d09bf2245d3b36131601a88, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:19,702 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=39abe5dfce1992816479263273d22d55, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:19,702 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290639702"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290639702"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290639702"}]},"ts":"1676290639702"} 2023-02-13 12:17:19,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 39abe5dfce1992816479263273d22d55, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:19,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:19,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 39abe5dfce1992816479263273d22d55, NAME => 'testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 12:17:19,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:17:19,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:19,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,856 INFO [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,857 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1 2023-02-13 12:17:19,857 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1 2023-02-13 12:17:19,857 INFO [StoreOpener-39abe5dfce1992816479263273d22d55-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39abe5dfce1992816479263273d22d55 columnFamilyName cf1 2023-02-13 12:17:19,869 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9-top 2023-02-13 12:17:19,873 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e-top 2023-02-13 12:17:19,877 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2-top 2023-02-13 12:17:19,880 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0-top 2023-02-13 12:17:19,883 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854-top 2023-02-13 12:17:19,886 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680-top 2023-02-13 12:17:19,889 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c-top 2023-02-13 12:17:19,889 INFO [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(310): Store=39abe5dfce1992816479263273d22d55/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:19,890 INFO [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,890 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2 2023-02-13 12:17:19,890 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2 2023-02-13 12:17:19,891 INFO [StoreOpener-39abe5dfce1992816479263273d22d55-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39abe5dfce1992816479263273d22d55 columnFamilyName cf2 2023-02-13 12:17:19,902 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2-top 2023-02-13 12:17:19,906 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387-top 2023-02-13 12:17:19,910 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df-top 2023-02-13 12:17:19,914 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3-top 2023-02-13 12:17:19,918 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd-top 2023-02-13 12:17:19,922 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5-top 2023-02-13 12:17:19,925 DEBUG [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e-top 2023-02-13 12:17:19,925 INFO [StoreOpener-39abe5dfce1992816479263273d22d55-1] regionserver.HStore(310): Store=39abe5dfce1992816479263273d22d55/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:19,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:17:19,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 39abe5dfce1992816479263273d22d55 2023-02-13 12:17:19,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 39abe5dfce1992816479263273d22d55; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73600972, jitterRate=0.09673994779586792}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:17:19,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 39abe5dfce1992816479263273d22d55: Running coprocessor pre-open hook at 1676290639855Writing region info on filesystem at 1676290639855Initializing all the Stores at 1676290639855Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290639856 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290639856Cleaning up temporary data from old regions at 1676290639927 (+71 ms)Cleaning up detritus from prior splits at 1676290639928 (+1 ms)Running coprocessor post-open hooks at 1676290639930 (+2 ms)Region opened successfully at 1676290639930 2023-02-13 12:17:19,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55., pid=118, masterSystemTime=1676290639851 2023-02-13 12:17:19,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:19,932 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:17:19,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:17:19,932 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:17:19,932 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:19,932 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:19,932 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1912): 39abe5dfce1992816479263273d22d55/cf2 is initiating minor compaction (all files) 2023-02-13 12:17:19,932 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1912): 39abe5dfce1992816479263273d22d55/cf1 is initiating minor compaction (all files) 2023-02-13 12:17:19,932 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 39abe5dfce1992816479263273d22d55/cf2 in testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:19,932 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 39abe5dfce1992816479263273d22d55/cf1 in testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:19,932 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e-top] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/.tmp, totalSize=34.9 K 2023-02-13 12:17:19,932 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2-top] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/.tmp, totalSize=34.9 K 2023-02-13 12:17:19,933 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676290634736 2023-02-13 12:17:19,933 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676290634736 2023-02-13 12:17:19,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:19,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:19,933 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676290634958 2023-02-13 12:17:19,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:19,933 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676290634958 2023-02-13 12:17:19,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f4e141a29d09bf2245d3b36131601a88, NAME => 'testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 12:17:19,933 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=39abe5dfce1992816479263273d22d55, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:19,934 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290639933"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290639933"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290639933"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290639933"}]},"ts":"1676290639933"} 2023-02-13 12:17:19,934 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676290635198 2023-02-13 12:17:19,934 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676290635198 2023-02-13 12:17:19,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:17:19,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:19,934 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676290635448 2023-02-13 12:17:19,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:19,934 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676290635448 2023-02-13 12:17:19,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:19,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:19,934 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676290635665 2023-02-13 12:17:19,934 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676290635665 2023-02-13 12:17:19,935 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676290635890 2023-02-13 12:17:19,935 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676290635890 2023-02-13 12:17:19,935 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676290636131 2023-02-13 12:17:19,935 INFO [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:19,935 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676290636131 2023-02-13 12:17:19,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-13 12:17:19,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 39abe5dfce1992816479263273d22d55, server=jenkins-hbase16.apache.org,33055,1676290533708 in 232 msec 2023-02-13 12:17:19,937 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1 2023-02-13 12:17:19,937 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1 2023-02-13 12:17:19,937 INFO [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f4e141a29d09bf2245d3b36131601a88 columnFamilyName cf1 2023-02-13 12:17:19,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=39abe5dfce1992816479263273d22d55, ASSIGN in 393 msec 2023-02-13 12:17:19,947 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] throttle.PressureAwareThroughputController(145): 39abe5dfce1992816479263273d22d55#cf2#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:17:19,948 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] throttle.PressureAwareThroughputController(145): 39abe5dfce1992816479263273d22d55#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:17:19,951 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9-bottom 2023-02-13 12:17:19,954 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e-bottom 2023-02-13 12:17:19,958 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2-bottom 2023-02-13 12:17:19,965 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0-bottom 2023-02-13 12:17:19,969 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854-bottom 2023-02-13 12:17:19,970 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/.tmp/cf1/6a0be3183b9741258f166c780eed1082 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/6a0be3183b9741258f166c780eed1082 2023-02-13 12:17:19,970 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/.tmp/cf2/572678b64429476ea81cf26bb5c2fc71 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/572678b64429476ea81cf26bb5c2fc71 2023-02-13 12:17:19,973 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680-bottom 2023-02-13 12:17:19,974 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 39abe5dfce1992816479263273d22d55/cf1 of 39abe5dfce1992816479263273d22d55 into 6a0be3183b9741258f166c780eed1082(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:17:19,974 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 39abe5dfce1992816479263273d22d55: 2023-02-13 12:17:19,974 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55., storeName=39abe5dfce1992816479263273d22d55/cf1, priority=9, startTime=1676290639931; duration=0sec 2023-02-13 12:17:19,974 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:19,974 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 39abe5dfce1992816479263273d22d55/cf2 of 39abe5dfce1992816479263273d22d55 into 572678b64429476ea81cf26bb5c2fc71(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:17:19,975 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 39abe5dfce1992816479263273d22d55: 2023-02-13 12:17:19,975 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55., storeName=39abe5dfce1992816479263273d22d55/cf2, priority=9, startTime=1676290639932; duration=0sec 2023-02-13 12:17:19,975 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:19,976 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c-bottom 2023-02-13 12:17:19,977 INFO [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(310): Store=f4e141a29d09bf2245d3b36131601a88/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:19,977 INFO [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:19,978 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2 2023-02-13 12:17:19,978 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2 2023-02-13 12:17:19,978 INFO [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f4e141a29d09bf2245d3b36131601a88 columnFamilyName cf2 2023-02-13 12:17:19,990 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2-bottom 2023-02-13 12:17:19,994 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387-bottom 2023-02-13 12:17:19,998 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df-bottom 2023-02-13 12:17:20,001 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3-bottom 2023-02-13 12:17:20,005 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd-bottom 2023-02-13 12:17:20,008 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5-bottom 2023-02-13 12:17:20,012 DEBUG [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e-bottom 2023-02-13 12:17:20,012 INFO [StoreOpener-f4e141a29d09bf2245d3b36131601a88-1] regionserver.HStore(310): Store=f4e141a29d09bf2245d3b36131601a88/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:20,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:20,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:20,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:17:20,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:20,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened f4e141a29d09bf2245d3b36131601a88; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59849971, jitterRate=-0.1081659346818924}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:17:20,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for f4e141a29d09bf2245d3b36131601a88: Running coprocessor pre-open hook at 1676290639934Writing region info on filesystem at 1676290639934Initializing all the Stores at 1676290639935 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290639935Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290639935Cleaning up temporary data from old regions at 1676290640015 (+80 ms)Cleaning up detritus from prior splits at 1676290640015Running coprocessor post-open hooks at 1676290640018 (+3 ms)Region opened successfully at 1676290640018 2023-02-13 12:17:20,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88., pid=117, masterSystemTime=1676290639851 2023-02-13 12:17:20,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:20,020 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:17:20,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:17:20,020 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:17:20,020 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:20,020 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1912): f4e141a29d09bf2245d3b36131601a88/cf1 is initiating minor compaction (all files) 2023-02-13 12:17:20,020 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:20,020 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1912): f4e141a29d09bf2245d3b36131601a88/cf2 is initiating minor compaction (all files) 2023-02-13 12:17:20,020 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f4e141a29d09bf2245d3b36131601a88/cf1 in testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:20,020 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f4e141a29d09bf2245d3b36131601a88/cf2 in testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:20,021 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e-bottom] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/.tmp, totalSize=34.9 K 2023-02-13 12:17:20,021 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2-bottom] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/.tmp, totalSize=34.9 K 2023-02-13 12:17:20,022 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290634736 2023-02-13 12:17:20,022 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290634736 2023-02-13 12:17:20,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:20,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:20,022 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290634958 2023-02-13 12:17:20,022 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=f4e141a29d09bf2245d3b36131601a88, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:20,022 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290634958 2023-02-13 12:17:20,022 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290640022"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290640022"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290640022"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290640022"}]},"ts":"1676290640022"} 2023-02-13 12:17:20,023 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290635198 2023-02-13 12:17:20,023 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290635198 2023-02-13 12:17:20,023 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290635448 2023-02-13 12:17:20,023 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290635448 2023-02-13 12:17:20,023 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290635665 2023-02-13 12:17:20,024 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290635665 2023-02-13 12:17:20,024 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290635890 2023-02-13 12:17:20,024 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290635890 2023-02-13 12:17:20,024 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting 05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290636131 2023-02-13 12:17:20,025 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting 1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290636131 2023-02-13 12:17:20,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-13 12:17:20,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure f4e141a29d09bf2245d3b36131601a88, server=jenkins-hbase16.apache.org,33055,1676290533708 in 324 msec 2023-02-13 12:17:20,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-13 12:17:20,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f4e141a29d09bf2245d3b36131601a88, ASSIGN in 483 msec 2023-02-13 12:17:20,028 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 12:17:20,028 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:20,028 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 12:17:20,034 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] throttle.PressureAwareThroughputController(145): f4e141a29d09bf2245d3b36131601a88#cf1#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:17:20,034 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] throttle.PressureAwareThroughputController(145): f4e141a29d09bf2245d3b36131601a88#cf2#compaction#111 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:17:20,044 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,044 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 12:17:20,045 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:20,045 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:20,049 ERROR [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 12:17:20,049 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 12:17:20,049 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 12:17:20,049 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 12:17:20,050 INFO [Listener at localhost.localdomain/46087] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:17:20,051 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 12:17:20,052 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-13 12:17:20,053 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 12:17:20,053 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 12:17:20,053 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 23] 2023-02-13 12:17:20,053 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-02-13 12:17:20,053 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 21] 2023-02-13 12:17:20,053 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/.tmp/cf2/e8b4de95f4a4412585be3324eadf3a37 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/e8b4de95f4a4412585be3324eadf3a37 2023-02-13 12:17:20,053 DEBUG [Listener at localhost.localdomain/46087] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-02-13 12:17:20,054 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 12:17:20,054 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 12:17:20,054 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 12:17:20,054 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:17:20,055 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:20,055 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b5548b9ced331770542633d75bc36e0d, daughterA=6479942123147c70630ae414ae269a27, daughterB=606b11da264cede32c10b7b752e6175e 2023-02-13 12:17:20,055 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:20,055 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 12:17:20,056 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:20,056 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=3bb9f1de2ea5395a2121754580ffb353, daughterA=35372d8b400d126b69effe8c26d81515, daughterB=a9394292bc62d1bc8ddb07cb3a18edde 2023-02-13 12:17:20,056 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:20,056 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 12:17:20,056 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/.tmp/cf1/3267d95d07534b6faa519b10da349ae3 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/3267d95d07534b6faa519b10da349ae3 2023-02-13 12:17:20,057 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:20,057 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c234839e44de2733ae92a93b34e9e56e, daughterA=5dc4f3767bed16f7e8e012be0cd512b5, daughterB=5188b35eb20e08a4c9da35109d932d6e 2023-02-13 12:17:20,057 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c234839e44de2733ae92a93b34e9e56e, ASSIGN 2023-02-13 12:17:20,057 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:20,057 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 12:17:20,058 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:20,058 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=47636dde92dbddaf4a0d879337d26302, daughterA=470a6381d0d0d2d0d7d5770c5e13dc82, daughterB=b46ec0870b7b2f348fd0578c2ec103c7 2023-02-13 12:17:20,058 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:20,058 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 12:17:20,058 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:20,059 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b4de9409ed4174da452e463dd5faeb66, daughterA=11f43b1e7c3649d8ac68d960818d7123, daughterB=b39af216aaba42ddb0ac2f5e3ecf0267 2023-02-13 12:17:20,059 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:20,059 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 12:17:20,059 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:20,059 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:20,060 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRollbackForSplitTableRegionWithReplica, region=07406c18ee535b7620fbe7fdd2bbe517 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=223dadc96dff0acc6b1c2fb058f22e1a, daughterA=d2c089f1da267b0a98812426b75c28c6, daughterB=245e2877581a3c4db2b8c0e908eaaf25 2023-02-13 12:17:20,060 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:20,060 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 12:17:20,060 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:20,060 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f4e141a29d09bf2245d3b36131601a88/cf2 of f4e141a29d09bf2245d3b36131601a88 into e8b4de95f4a4412585be3324eadf3a37(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:17:20,060 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:20,061 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f4e141a29d09bf2245d3b36131601a88: 2023-02-13 12:17:20,061 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 12:17:20,061 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88., storeName=f4e141a29d09bf2245d3b36131601a88/cf2, priority=9, startTime=1676290640020; duration=0sec 2023-02-13 12:17:20,061 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:20,061 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:20,062 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f4e141a29d09bf2245d3b36131601a88/cf1 of f4e141a29d09bf2245d3b36131601a88 into 3267d95d07534b6faa519b10da349ae3(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:17:20,063 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f4e141a29d09bf2245d3b36131601a88: 2023-02-13 12:17:20,063 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88., storeName=f4e141a29d09bf2245d3b36131601a88/cf1, priority=9, startTime=1676290640019; duration=0sec 2023-02-13 12:17:20,063 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:20,064 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,064 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN 2023-02-13 12:17:20,064 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:20,064 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f4e141a29d09bf2245d3b36131601a88, ASSIGN 2023-02-13 12:17:20,065 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=39abe5dfce1992816479263273d22d55, ASSIGN 2023-02-13 12:17:20,065 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure f4e141a29d09bf2245d3b36131601a88, server=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:20,065 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 39abe5dfce1992816479263273d22d55, server=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:20,065 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 12:17:20,065 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,065 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,065 INFO [Listener at localhost.localdomain/46087] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,065 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:20,065 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0820811d7bb9089a2330649ab2872528, server=jenkins-hbase16.apache.org,33055,1676290533708 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:20,066 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f4e141a29d09bf2245d3b36131601a88, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:20,066 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=39abe5dfce1992816479263273d22d55, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:20,066 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure f4e141a29d09bf2245d3b36131601a88, server=jenkins-hbase16.apache.org,33055,1676290533708 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:20,066 DEBUG [Listener at localhost.localdomain/46087] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 39abe5dfce1992816479263273d22d55, server=jenkins-hbase16.apache.org,33055,1676290533708 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 12:17:20,066 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 15 msec 2023-02-13 12:17:20,068 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 12:17:20,068 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759, table=hbase:meta, region=1588230740 2023-02-13 12:17:20,069 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676290670069 2023-02-13 12:17:20,071 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,071 DEBUG [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:17:20,074 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=97396b7f803d52a54325415167dadd83, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:20,074 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=0820811d7bb9089a2330649ab2872528, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=2 2023-02-13 12:17:20,075 INFO [Listener at localhost.localdomain/46087] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase16.apache.org,33055,1676290533708, table=testRecoveryAndDoubleExecution, region=0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,075 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=f4e141a29d09bf2245d3b36131601a88, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=86 2023-02-13 12:17:20,075 DEBUG [Listener at localhost.localdomain/46087] assignment.RegionStateStore(147): Load hbase:meta entry region=39abe5dfce1992816479263273d22d55, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,33055,1676290533708, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708, openSeqNum=86 2023-02-13 12:17:20,075 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 12:17:20,075 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676290700075 2023-02-13 12:17:20,075 INFO [Listener at localhost.localdomain/46087] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676290760075 2023-02-13 12:17:20,076 INFO [Listener at localhost.localdomain/46087] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 12:17:20,076 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 12:17:20,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0820811d7bb9089a2330649ab2872528, daughterA=f4e141a29d09bf2245d3b36131601a88, daughterB=39abe5dfce1992816479263273d22d55 in 3.6870 sec 2023-02-13 12:17:20,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33055] ipc.CallRunner(144): callId: 860 service: ClientService methodName: Scan size: 154 connection: 188.40.62.62:45960 deadline: 1676290700327, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528. is not online on jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:20,438 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 12:17:20,445 INFO [Listener at localhost.localdomain/46087] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 12:17:20,448 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 12:17:20,460 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:17:20,461 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-13 12:17:20,461 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testRecoveryAndDoubleExecution 2023-02-13 12:17:20,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:20,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 12:17:20,464 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290640464"}]},"ts":"1676290640464"} 2023-02-13 12:17:20,465 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 12:17:20,505 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-13 12:17:20,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f4e141a29d09bf2245d3b36131601a88, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=39abe5dfce1992816479263273d22d55, UNASSIGN}] 2023-02-13 12:17:20,507 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=39abe5dfce1992816479263273d22d55, UNASSIGN 2023-02-13 12:17:20,507 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=39abe5dfce1992816479263273d22d55, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:20,508 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290640507"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290640507"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290640507"}]},"ts":"1676290640507"} 2023-02-13 12:17:20,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 39abe5dfce1992816479263273d22d55, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:20,511 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f4e141a29d09bf2245d3b36131601a88, UNASSIGN 2023-02-13 12:17:20,511 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=f4e141a29d09bf2245d3b36131601a88, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:20,511 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290640511"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290640511"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290640511"}]},"ts":"1676290640511"} 2023-02-13 12:17:20,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure f4e141a29d09bf2245d3b36131601a88, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:20,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 12:17:20,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:20,661 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing f4e141a29d09bf2245d3b36131601a88, disabling compactions & flushes 2023-02-13 12:17:20,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:20,661 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:20,661 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. after waiting 0 ms 2023-02-13 12:17:20,661 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:20,662 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e-bottom] to archive 2023-02-13 12:17:20,662 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:17:20,664 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,665 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,667 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,668 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,669 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,670 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,671 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,673 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2-bottom] to archive 2023-02-13 12:17:20,673 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:17:20,675 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,676 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,677 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,678 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,679 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,679 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,680 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 12:17:20,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:20,684 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88. 2023-02-13 12:17:20,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for f4e141a29d09bf2245d3b36131601a88: Waiting for close lock at 1676290640661Running coprocessor pre-close hooks at 1676290640661Disabling compacts and flushes for region at 1676290640661Disabling writes for close at 1676290640661Writing region close event to WAL at 1676290640681 (+20 ms)Running coprocessor post-close hooks at 1676290640684 (+3 ms)Closed at 1676290640684 2023-02-13 12:17:20,686 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:20,686 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 39abe5dfce1992816479263273d22d55, disabling compactions & flushes 2023-02-13 12:17:20,686 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:20,686 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=f4e141a29d09bf2245d3b36131601a88, regionState=CLOSED 2023-02-13 12:17:20,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:20,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. after waiting 0 ms 2023-02-13 12:17:20,686 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290640686"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290640686"}]},"ts":"1676290640686"} 2023-02-13 12:17:20,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:20,688 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e-top] to archive 2023-02-13 12:17:20,689 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:17:20,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-13 12:17:20,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure f4e141a29d09bf2245d3b36131601a88, server=jenkins-hbase16.apache.org,33055,1676290533708 in 176 msec 2023-02-13 12:17:20,690 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f4e141a29d09bf2245d3b36131601a88, UNASSIGN in 184 msec 2023-02-13 12:17:20,691 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,692 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,693 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,694 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,695 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,696 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,698 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2-top] to archive 2023-02-13 12:17:20,698 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:17:20,700 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,701 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,702 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,703 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,703 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,704 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,705 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 12:17:20,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:20,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55. 2023-02-13 12:17:20,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 39abe5dfce1992816479263273d22d55: Waiting for close lock at 1676290640686Running coprocessor pre-close hooks at 1676290640686Disabling compacts and flushes for region at 1676290640686Disabling writes for close at 1676290640687 (+1 ms)Writing region close event to WAL at 1676290640706 (+19 ms)Running coprocessor post-close hooks at 1676290640709 (+3 ms)Closed at 1676290640709 2023-02-13 12:17:20,711 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,711 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=39abe5dfce1992816479263273d22d55, regionState=CLOSED 2023-02-13 12:17:20,711 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676290640711"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290640711"}]},"ts":"1676290640711"} 2023-02-13 12:17:20,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-13 12:17:20,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 39abe5dfce1992816479263273d22d55, server=jenkins-hbase16.apache.org,33055,1676290533708 in 203 msec 2023-02-13 12:17:20,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-02-13 12:17:20,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=39abe5dfce1992816479263273d22d55, UNASSIGN in 208 msec 2023-02-13 12:17:20,715 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290640715"}]},"ts":"1676290640715"} 2023-02-13 12:17:20,716 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 12:17:20,722 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-13 12:17:20,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 262 msec 2023-02-13 12:17:20,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 12:17:20,767 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-13 12:17:20,768 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testRecoveryAndDoubleExecution 2023-02-13 12:17:20,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:20,770 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:20,771 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:20,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 12:17:20,776 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,776 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:20,776 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,777 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/recovered.edits] 2023-02-13 12:17:20,778 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/recovered.edits] 2023-02-13 12:17:20,778 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/recovered.edits] 2023-02-13 12:17:20,780 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/3267d95d07534b6faa519b10da349ae3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf1/3267d95d07534b6faa519b10da349ae3 2023-02-13 12:17:20,780 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/.regioninfo to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/.regioninfo 2023-02-13 12:17:20,780 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/0c08830661364a319f483baf2fbce4a9 2023-02-13 12:17:20,781 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/1c5d02981368482ba2d2d33ba702a26e 2023-02-13 12:17:20,782 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/e8b4de95f4a4412585be3324eadf3a37 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/cf2/e8b4de95f4a4412585be3324eadf3a37 2023-02-13 12:17:20,782 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,782 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/21932edcea39499089bb0178bc96fcf2 2023-02-13 12:17:20,783 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/5e7fe42c78644940a6be41ff9c2ebdc0 2023-02-13 12:17:20,783 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,784 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/.regioninfo to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/.regioninfo 2023-02-13 12:17:20,785 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,785 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/a267453befc043d18fb0816dd2c95854 2023-02-13 12:17:20,786 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,786 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/df9cb29ea40640b794d94d4e7c7da680 2023-02-13 12:17:20,786 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/0c08830661364a319f483baf2fbce4a9.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,787 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,787 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf1/f862226517df4fd98f78ba440e75881c 2023-02-13 12:17:20,787 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/1c5d02981368482ba2d2d33ba702a26e.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,788 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,788 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/21932edcea39499089bb0178bc96fcf2.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,789 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,789 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/5e7fe42c78644940a6be41ff9c2ebdc0.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,789 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/05b6753f918044d1833b071aabfddeb2 2023-02-13 12:17:20,790 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/a267453befc043d18fb0816dd2c95854.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,790 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/2b3c57f7e47b4fcd94e2b424dad0c387 2023-02-13 12:17:20,791 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,792 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/df9cb29ea40640b794d94d4e7c7da680.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,792 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/4738c514b09f438187452c4befd8d3df 2023-02-13 12:17:20,792 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,794 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf1/f862226517df4fd98f78ba440e75881c.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,794 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/7f1af891ca3a444592fcac07a55d29a3 2023-02-13 12:17:20,794 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,796 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd 2023-02-13 12:17:20,796 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,797 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,797 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/b70e7918f9e9496c863a4b2dc588b4a5 2023-02-13 12:17:20,798 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/05b6753f918044d1833b071aabfddeb2.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,799 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/cf2/c58bf212ff804449a1fded2bbeebed5e 2023-02-13 12:17:20,799 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,799 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/2b3c57f7e47b4fcd94e2b424dad0c387.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,800 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/39abe5dfce1992816479263273d22d55/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,801 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/4738c514b09f438187452c4befd8d3df.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,801 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/recovered.edits/85.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528/recovered.edits/85.seqid 2023-02-13 12:17:20,802 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,802 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/7f1af891ca3a444592fcac07a55d29a3.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,803 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/6a0be3183b9741258f166c780eed1082 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf1/6a0be3183b9741258f166c780eed1082 2023-02-13 12:17:20,803 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/a7eb5d3fc9cc47339b7b3f2a6fa6e3dd.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,804 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/b70e7918f9e9496c863a4b2dc588b4a5.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,804 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/572678b64429476ea81cf26bb5c2fc71 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/cf2/572678b64429476ea81cf26bb5c2fc71 2023-02-13 12:17:20,805 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/f4e141a29d09bf2245d3b36131601a88/cf2/c58bf212ff804449a1fded2bbeebed5e.0820811d7bb9089a2330649ab2872528 2023-02-13 12:17:20,806 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/recovered.edits/90.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55/recovered.edits/90.seqid 2023-02-13 12:17:20,806 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/recovered.edits/90.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88/recovered.edits/90.seqid 2023-02-13 12:17:20,807 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/39abe5dfce1992816479263273d22d55 2023-02-13 12:17:20,807 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testRecoveryAndDoubleExecution/f4e141a29d09bf2245d3b36131601a88 2023-02-13 12:17:20,807 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 12:17:20,809 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:20,810 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-13 12:17:20,812 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-13 12:17:20,813 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:20,813 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-13 12:17:20,813 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290640813"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:20,813 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290640813"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:20,813 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290640813"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:20,815 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 12:17:20,815 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0820811d7bb9089a2330649ab2872528, NAME => 'testRecoveryAndDoubleExecution,,1676290634232.0820811d7bb9089a2330649ab2872528.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => f4e141a29d09bf2245d3b36131601a88, NAME => 'testRecoveryAndDoubleExecution,,1676290636389.f4e141a29d09bf2245d3b36131601a88.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 39abe5dfce1992816479263273d22d55, NAME => 'testRecoveryAndDoubleExecution,41,1676290636389.39abe5dfce1992816479263273d22d55.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 12:17:20,815 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-13 12:17:20,815 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676290640815"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:20,817 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-13 12:17:20,848 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 12:17:20,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 80 msec 2023-02-13 12:17:20,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 12:17:20,873 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-13 12:17:20,886 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=377 (was 369) - Thread LEAK? -, OpenFileDescriptor=841 (was 747) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=247 (was 229) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=5407 (was 5422) 2023-02-13 12:17:20,898 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377, OpenFileDescriptor=841, MaxFileDescriptor=60000, SystemLoadAverage=247, ProcessCount=167, AvailableMemoryMB=5406 2023-02-13 12:17:20,898 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:17:20,905 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 12:17:20,906 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 12:17:20,907 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 12:17:20,909 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:17:20,909 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:17:20,911 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:20,911 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2 empty. 2023-02-13 12:17:20,912 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:20,912 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 12:17:20,920 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 12:17:20,921 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 601e1d5b9375e4ae31702b41106955a2, NAME => 'testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:17:20,927 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:20,927 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 601e1d5b9375e4ae31702b41106955a2, disabling compactions & flushes 2023-02-13 12:17:20,927 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:20,927 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:20,927 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. after waiting 0 ms 2023-02-13 12:17:20,928 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:20,928 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:20,928 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 601e1d5b9375e4ae31702b41106955a2: Waiting for close lock at 1676290640927Disabling compacts and flushes for region at 1676290640927Disabling writes for close at 1676290640928 (+1 ms)Writing region close event to WAL at 1676290640928Closed at 1676290640928 2023-02-13 12:17:20,929 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:17:20,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290640930"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290640930"}]},"ts":"1676290640930"} 2023-02-13 12:17:20,931 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:17:20,931 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:17:20,931 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290640931"}]},"ts":"1676290640931"} 2023-02-13 12:17:20,932 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-13 12:17:20,947 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:17:20,948 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:17:20,948 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:17:20,948 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:17:20,948 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:17:20,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=601e1d5b9375e4ae31702b41106955a2, ASSIGN}] 2023-02-13 12:17:20,949 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=601e1d5b9375e4ae31702b41106955a2, ASSIGN 2023-02-13 12:17:20,950 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=601e1d5b9375e4ae31702b41106955a2, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:17:21,100 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:17:21,101 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=601e1d5b9375e4ae31702b41106955a2, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,101 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290641101"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290641101"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290641101"}]},"ts":"1676290641101"} 2023-02-13 12:17:21,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 601e1d5b9375e4ae31702b41106955a2, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:21,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:21,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 601e1d5b9375e4ae31702b41106955a2, NAME => 'testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:17:21,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:17:21,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:21,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:21,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:21,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:21,259 INFO [StoreOpener-601e1d5b9375e4ae31702b41106955a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:21,260 DEBUG [StoreOpener-601e1d5b9375e4ae31702b41106955a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1 2023-02-13 12:17:21,260 DEBUG [StoreOpener-601e1d5b9375e4ae31702b41106955a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1 2023-02-13 12:17:21,261 INFO [StoreOpener-601e1d5b9375e4ae31702b41106955a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 601e1d5b9375e4ae31702b41106955a2 columnFamilyName cf1 2023-02-13 12:17:21,261 INFO [StoreOpener-601e1d5b9375e4ae31702b41106955a2-1] regionserver.HStore(310): Store=601e1d5b9375e4ae31702b41106955a2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:21,261 INFO [StoreOpener-601e1d5b9375e4ae31702b41106955a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:21,262 DEBUG [StoreOpener-601e1d5b9375e4ae31702b41106955a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2 2023-02-13 12:17:21,263 DEBUG [StoreOpener-601e1d5b9375e4ae31702b41106955a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2 2023-02-13 12:17:21,263 INFO [StoreOpener-601e1d5b9375e4ae31702b41106955a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 601e1d5b9375e4ae31702b41106955a2 columnFamilyName cf2 2023-02-13 12:17:21,263 INFO [StoreOpener-601e1d5b9375e4ae31702b41106955a2-1] regionserver.HStore(310): Store=601e1d5b9375e4ae31702b41106955a2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:21,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:21,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:21,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:17:21,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:21,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:17:21,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 601e1d5b9375e4ae31702b41106955a2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61536321, jitterRate=-0.0830373615026474}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:17:21,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 601e1d5b9375e4ae31702b41106955a2: Running coprocessor pre-open hook at 1676290641258Writing region info on filesystem at 1676290641258Initializing all the Stores at 1676290641259 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290641259Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290641259Cleaning up temporary data from old regions at 1676290641265 (+6 ms)Cleaning up detritus from prior splits at 1676290641265Running coprocessor post-open hooks at 1676290641269 (+4 ms)Region opened successfully at 1676290641269 2023-02-13 12:17:21,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2., pid=127, masterSystemTime=1676290641255 2023-02-13 12:17:21,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:21,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:21,272 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=601e1d5b9375e4ae31702b41106955a2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,272 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290641272"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290641272"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290641272"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290641272"}]},"ts":"1676290641272"} 2023-02-13 12:17:21,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-13 12:17:21,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 601e1d5b9375e4ae31702b41106955a2, server=jenkins-hbase16.apache.org,33055,1676290533708 in 170 msec 2023-02-13 12:17:21,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-13 12:17:21,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=601e1d5b9375e4ae31702b41106955a2, ASSIGN in 327 msec 2023-02-13 12:17:21,277 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:17:21,277 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290641277"}]},"ts":"1676290641277"} 2023-02-13 12:17:21,278 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-13 12:17:21,289 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:17:21,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 384 msec 2023-02-13 12:17:21,415 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:21,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,417 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:21,417 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,417 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 12:17:21,417 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 12:17:21,417 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,417 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,422 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,422 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,422 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,422 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,423 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,423 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,424 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:21,424 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,424 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,424 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:21,424 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:21,424 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:21,424 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,424 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:21,424 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:21,424 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:21,424 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:21,424 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,425 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:21,425 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,425 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:21,425 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:21,425 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:21,425 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:21,425 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. started... 2023-02-13 12:17:21,426 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 601e1d5b9375e4ae31702b41106955a2 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 12:17:21,430 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,433 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/cb3984de5d044e76869ea720632959e3 2023-02-13 12:17:21,439 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,439 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,439 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:21,439 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,439 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:21,444 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/9ea3dcb194954417a5194c616b9a77e8 2023-02-13 12:17:21,447 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/cb3984de5d044e76869ea720632959e3 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/cb3984de5d044e76869ea720632959e3 2023-02-13 12:17:21,450 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/cb3984de5d044e76869ea720632959e3, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:17:21,451 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/9ea3dcb194954417a5194c616b9a77e8 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/9ea3dcb194954417a5194c616b9a77e8 2023-02-13 12:17:21,453 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/9ea3dcb194954417a5194c616b9a77e8, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:17:21,454 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 601e1d5b9375e4ae31702b41106955a2 in 28ms, sequenceid=6, compaction requested=false 2023-02-13 12:17:21,454 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:21,455 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 601e1d5b9375e4ae31702b41106955a2: 2023-02-13 12:17:21,455 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:21,455 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:21,455 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:21,455 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,455 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:21,455 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:21,464 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,464 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:21,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:21,464 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,464 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:21,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:21,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 12:17:21,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17e3d072[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:21,466 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 12:17:21,466 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,472 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,472 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,472 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,472 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,472 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:21,472 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:21,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,472 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:21,472 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:21,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,473 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:21,473 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:21,473 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:21,473 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:21,473 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:21,473 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:21,480 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:21,480 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,480 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:21,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:21,481 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:21,489 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:21,489 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:21,489 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:21,489 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,489 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:21,489 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:21,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:21,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:21,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:21,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:17:21,491 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 12:17:21,491 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:21,491 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:21,491 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,491 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:21,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,497 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,497 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:21,498 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,498 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:21,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,499 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,499 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:21,500 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,529 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,529 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,529 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,529 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,529 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,529 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,529 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:21,529 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:21,529 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,529 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,529 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,529 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,530 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:21,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,530 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:21,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,530 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,530 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,530 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,530 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,530 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,530 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,530 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,530 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,630 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:21,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:21,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:21,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:21,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,643 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:21,643 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,643 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 12:17:21,643 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 12:17:21,643 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,643 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,655 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,655 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,655 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,655 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,656 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,656 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:21,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,657 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:21,657 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,657 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,657 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,657 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:21,657 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,657 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:21,657 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,657 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:21,657 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:21,657 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:21,657 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:21,657 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:21,657 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,658 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:21,658 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:21,658 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:21,658 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:21,658 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:21,658 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:21,658 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. started... 2023-02-13 12:17:21,658 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 601e1d5b9375e4ae31702b41106955a2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:21,663 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,665 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/198dab6d84724bc1965f0118b668b48f 2023-02-13 12:17:21,672 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,672 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,672 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:21,672 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,672 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:21,674 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/3d3d46435f4f47edb556d730becdc676 2023-02-13 12:17:21,678 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/198dab6d84724bc1965f0118b668b48f as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/198dab6d84724bc1965f0118b668b48f 2023-02-13 12:17:21,681 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/198dab6d84724bc1965f0118b668b48f, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:17:21,682 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/3d3d46435f4f47edb556d730becdc676 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/3d3d46435f4f47edb556d730becdc676 2023-02-13 12:17:21,685 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/3d3d46435f4f47edb556d730becdc676, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:17:21,686 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 601e1d5b9375e4ae31702b41106955a2 in 28ms, sequenceid=19, compaction requested=false 2023-02-13 12:17:21,686 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 601e1d5b9375e4ae31702b41106955a2: 2023-02-13 12:17:21,686 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:21,686 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:21,686 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:21,686 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,686 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:21,686 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:21,697 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,697 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:21,697 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,697 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:21,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 12:17:21,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22161c66[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:21,699 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 12:17:21,699 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,705 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,705 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,705 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,706 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,706 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:21,706 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:21,706 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:21,706 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:21,706 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:21,706 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:21,706 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:21,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,706 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:21,706 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:21,706 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:21,714 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:21,714 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,714 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:21,722 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:21,722 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,722 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,722 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:21,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:21,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:21,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:21,730 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:21,730 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,731 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:21,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:17:21,733 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 12:17:21,733 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:21,733 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:21,733 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,733 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:21,739 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:21,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,739 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,739 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,740 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,741 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,741 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,741 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,747 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:21,747 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:21,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,747 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter because node does not exist (not an error) 2023-02-13 12:17:21,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,747 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,747 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:21,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,747 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,748 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:21,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,748 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,748 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,748 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,748 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,748 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,748 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,748 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,748 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,749 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,848 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:21,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:21,859 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:21,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:21,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,860 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:21,860 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,860 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 12:17:21,860 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 12:17:21,861 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,861 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,870 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,870 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,870 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,870 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,870 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,870 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:21,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:21,871 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,871 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:21,871 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,871 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:21,871 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:21,871 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:21,872 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:21,872 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:21,872 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,872 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,872 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:21,872 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:21,872 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,872 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:21,872 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:21,872 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:21,872 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:21,872 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:21,872 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:21,872 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:21,872 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:21,872 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. started... 2023-02-13 12:17:21,872 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 601e1d5b9375e4ae31702b41106955a2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:21,880 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,884 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/026c94d1e64c429ebf09713989f9af36 2023-02-13 12:17:21,898 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/ecafb8b2189b4e04a9c161d5db2162f6 2023-02-13 12:17:21,902 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/026c94d1e64c429ebf09713989f9af36 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/026c94d1e64c429ebf09713989f9af36 2023-02-13 12:17:21,904 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,904 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,904 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:21,905 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,905 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:21,906 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/026c94d1e64c429ebf09713989f9af36, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:17:21,907 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/ecafb8b2189b4e04a9c161d5db2162f6 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/ecafb8b2189b4e04a9c161d5db2162f6 2023-02-13 12:17:21,911 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/ecafb8b2189b4e04a9c161d5db2162f6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:17:21,911 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 601e1d5b9375e4ae31702b41106955a2 in 39ms, sequenceid=32, compaction requested=true 2023-02-13 12:17:21,911 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 601e1d5b9375e4ae31702b41106955a2: 2023-02-13 12:17:21,912 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:21,912 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:21,912 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:21,912 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,912 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:21,912 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:21,922 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,922 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:21,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:21,922 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,922 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:21,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:21,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:21,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:21,924 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 12:17:21,924 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9a54aa4[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:21,924 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 12:17:21,924 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,930 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,930 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,930 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,931 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,931 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:21,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,931 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:21,931 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:21,931 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:21,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,931 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:21,931 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:21,931 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:21,931 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:21,931 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:21,931 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:21,939 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:21,939 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,939 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:21,947 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:21,947 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,947 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,947 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:21,947 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,947 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:21,947 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:21,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:21,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:21,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:21,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:21,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:21,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:21,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:17:21,950 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 12:17:21,950 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:21,950 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:21,950 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,950 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:21,955 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,955 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,955 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:21,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:21,955 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,955 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,955 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:21,956 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,956 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,956 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:21,956 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:21,957 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,958 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,958 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,964 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:21,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:21,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:21,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:21,964 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:21,964 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:21,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:21,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,964 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:21,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:21,965 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,965 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:21,965 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,965 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:21,965 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:21,965 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,965 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:21,965 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:21,965 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:21,965 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,065 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:22,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:22,076 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:22,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:22,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,078 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:22,078 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,078 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 12:17:22,078 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 12:17:22,078 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,078 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,089 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,089 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,089 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,089 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,089 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,089 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:22,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,091 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:22,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,091 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:22,091 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:22,091 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:22,091 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. started... 2023-02-13 12:17:22,091 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 601e1d5b9375e4ae31702b41106955a2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:22,097 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,097 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,097 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,097 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,097 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:22,097 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:22,102 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/1fb43537a5264935b328ccb20628a27e 2023-02-13 12:17:22,111 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/dfa08e52d7b44a4393da3df1bee03ea3 2023-02-13 12:17:22,115 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/1fb43537a5264935b328ccb20628a27e as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/1fb43537a5264935b328ccb20628a27e 2023-02-13 12:17:22,118 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/1fb43537a5264935b328ccb20628a27e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:17:22,119 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/dfa08e52d7b44a4393da3df1bee03ea3 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/dfa08e52d7b44a4393da3df1bee03ea3 2023-02-13 12:17:22,122 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/dfa08e52d7b44a4393da3df1bee03ea3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:17:22,123 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 601e1d5b9375e4ae31702b41106955a2 in 32ms, sequenceid=45, compaction requested=true 2023-02-13 12:17:22,123 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 601e1d5b9375e4ae31702b41106955a2: 2023-02-13 12:17:22,123 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:22,123 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:22,123 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:22,123 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,123 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:22,124 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:22,130 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,130 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:22,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:22,131 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,131 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:22,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:22,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:22,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:22,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 12:17:22,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e0e5bf0[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:22,133 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 12:17:22,133 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,139 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,139 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,139 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,139 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,139 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:22,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,139 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:22,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,139 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:22,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,139 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:22,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,139 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:22,139 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:22,139 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:22,139 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:22,139 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:22,139 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:22,147 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:22,147 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,147 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:22,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:22,147 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:22,155 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:22,155 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,155 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:22,164 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:22,164 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,164 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:22,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:22,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:22,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:22,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:22,167 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:17:22,167 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 12:17:22,167 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:22,168 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:22,168 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,168 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:22,178 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,178 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,178 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,178 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,178 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,178 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,179 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:22,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:22,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,178 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,179 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:22,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,179 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:22,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,181 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:22,181 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,181 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,239 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,239 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,239 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,239 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,239 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,239 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,239 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,239 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,239 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,239 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:22,240 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:22,239 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,239 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,240 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,240 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:22,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,240 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:22,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,240 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,240 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,240 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,240 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,240 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,240 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,340 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:22,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:22,351 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:22,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:22,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,353 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:22,353 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,353 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 12:17:22,353 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 12:17:22,353 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,353 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,367 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,367 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,367 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,367 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,367 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,367 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:22,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:22,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:22,369 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:22,368 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:22,369 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:22,369 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:22,369 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:22,369 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:22,369 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. started... 2023-02-13 12:17:22,369 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 601e1d5b9375e4ae31702b41106955a2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:22,375 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,377 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/c48077b53ddb4b89b5911e872a851b79 2023-02-13 12:17:22,383 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,383 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,383 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:22,383 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,383 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:22,387 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/bc6eee85e4714e168ee9c8d17c42782c 2023-02-13 12:17:22,391 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/c48077b53ddb4b89b5911e872a851b79 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/c48077b53ddb4b89b5911e872a851b79 2023-02-13 12:17:22,395 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/c48077b53ddb4b89b5911e872a851b79, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:17:22,396 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/bc6eee85e4714e168ee9c8d17c42782c as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/bc6eee85e4714e168ee9c8d17c42782c 2023-02-13 12:17:22,399 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/bc6eee85e4714e168ee9c8d17c42782c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:17:22,400 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 601e1d5b9375e4ae31702b41106955a2 in 31ms, sequenceid=58, compaction requested=true 2023-02-13 12:17:22,400 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 601e1d5b9375e4ae31702b41106955a2: 2023-02-13 12:17:22,400 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:22,400 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:22,400 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:22,400 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,400 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:22,400 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:22,408 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,408 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:22,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:22,409 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,409 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:22,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:22,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:22,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:22,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 12:17:22,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17676803[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:22,411 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 12:17:22,411 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,416 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,416 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,416 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,417 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,417 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:22,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,417 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:22,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,417 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:22,417 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:22,417 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:22,417 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:22,417 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:22,417 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:22,417 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:22,417 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:22,425 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:22,425 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,425 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:22,433 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:22,433 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,433 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:22,433 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,433 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:22,434 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,434 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:22,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:22,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:22,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:22,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:22,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:22,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:22,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:17:22,437 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 12:17:22,437 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:22,437 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:22,437 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,437 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:22,441 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,441 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:22,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:22,441 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,441 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,442 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,441 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:22,442 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,442 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,442 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:22,442 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:22,444 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,444 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,444 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,450 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,450 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,450 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,450 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,450 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:22,450 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:22,450 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,450 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,450 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,450 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:22,450 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:22,450 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,450 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,451 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,451 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,451 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,451 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,451 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,451 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,451 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,451 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,451 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,551 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:22,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:22,561 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:22,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:22,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,563 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:22,563 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,563 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 12:17:22,563 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 12:17:22,563 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,563 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,574 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,574 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,574 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,574 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,574 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,574 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:22,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:22,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:22,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:22,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:22,576 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:22,576 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:22,576 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. started... 2023-02-13 12:17:22,576 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 601e1d5b9375e4ae31702b41106955a2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:22,584 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/4a1f71a232c24bb49de6e98e8f879cff 2023-02-13 12:17:22,594 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/0eac841ef3904705a610ef72d9aa87b7 2023-02-13 12:17:22,597 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/4a1f71a232c24bb49de6e98e8f879cff as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/4a1f71a232c24bb49de6e98e8f879cff 2023-02-13 12:17:22,600 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/4a1f71a232c24bb49de6e98e8f879cff, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:17:22,601 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/0eac841ef3904705a610ef72d9aa87b7 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/0eac841ef3904705a610ef72d9aa87b7 2023-02-13 12:17:22,604 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/0eac841ef3904705a610ef72d9aa87b7, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:17:22,605 INFO [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 601e1d5b9375e4ae31702b41106955a2 in 29ms, sequenceid=71, compaction requested=true 2023-02-13 12:17:22,605 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 601e1d5b9375e4ae31702b41106955a2: 2023-02-13 12:17:22,605 DEBUG [rs(jenkins-hbase16.apache.org,33055,1676290533708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:22,605 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:22,605 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:22,605 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,605 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 12:17:22,605 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 12:17:22,607 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,607 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,616 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,616 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:22,616 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,617 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:22,616 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,616 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:22,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:22,617 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,617 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:22,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:22,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:22,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:22,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 12:17:22,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@500a3a5[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:22,619 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 12:17:22,619 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,625 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,625 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,625 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,625 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:22,625 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,625 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:22,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,625 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:22,625 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:22,625 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:22,625 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:22,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,625 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:22,626 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 12:17:22,626 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 12:17:22,626 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:22,633 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:22,633 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,633 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:22,633 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,633 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:22,633 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:22,641 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 12:17:22,642 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,642 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 12:17:22,642 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,33055,1676290533708': 2023-02-13 12:17:22,645 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 12:17:22,645 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:22,645 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:22,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 12:17:22,645 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,645 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:22,650 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,650 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,650 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,651 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,651 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,652 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:22,652 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,652 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,658 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,658 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,658 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,658 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,658 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,658 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,658 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,658 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 12:17:22,659 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:22,658 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:22,658 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:22,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:22,659 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:22,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:22,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:22,659 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:22,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,659 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,660 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 12:17:22,760 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:22,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:22,767 DEBUG [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,768 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=601e1d5b9375e4ae31702b41106955a2, daughterA=52d2efcd0f602bb990ec308869fa8e25, daughterB=7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:22,769 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=601e1d5b9375e4ae31702b41106955a2, daughterA=52d2efcd0f602bb990ec308869fa8e25, daughterB=7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:22,769 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=601e1d5b9375e4ae31702b41106955a2, daughterA=52d2efcd0f602bb990ec308869fa8e25, daughterB=7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:22,769 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=601e1d5b9375e4ae31702b41106955a2, daughterA=52d2efcd0f602bb990ec308869fa8e25, daughterB=7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:22,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=601e1d5b9375e4ae31702b41106955a2, UNASSIGN}] 2023-02-13 12:17:22,771 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=601e1d5b9375e4ae31702b41106955a2, UNASSIGN 2023-02-13 12:17:22,772 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=601e1d5b9375e4ae31702b41106955a2, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:22,772 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290642772"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290642772"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290642772"}]},"ts":"1676290642772"} 2023-02-13 12:17:22,773 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 601e1d5b9375e4ae31702b41106955a2, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:22,881 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:17:22,924 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,924 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 601e1d5b9375e4ae31702b41106955a2, disabling compactions & flushes 2023-02-13 12:17:22,924 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:22,924 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:22,925 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. after waiting 0 ms 2023-02-13 12:17:22,925 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:22,925 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 601e1d5b9375e4ae31702b41106955a2 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 12:17:22,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/da243e4d8bfb4ca2a4b48cf2ac903455 2023-02-13 12:17:22,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/1e4f826918c0404faf02976d4b614d2c 2023-02-13 12:17:22,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf1/da243e4d8bfb4ca2a4b48cf2ac903455 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/da243e4d8bfb4ca2a4b48cf2ac903455 2023-02-13 12:17:22,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/da243e4d8bfb4ca2a4b48cf2ac903455, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:17:22,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.tmp/cf2/1e4f826918c0404faf02976d4b614d2c as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/1e4f826918c0404faf02976d4b614d2c 2023-02-13 12:17:22,965 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/1e4f826918c0404faf02976d4b614d2c, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:17:22,966 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 601e1d5b9375e4ae31702b41106955a2 in 41ms, sequenceid=82, compaction requested=true 2023-02-13 12:17:22,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 12:17:22,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:22,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. 2023-02-13 12:17:22,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 601e1d5b9375e4ae31702b41106955a2: Waiting for close lock at 1676290642924Running coprocessor pre-close hooks at 1676290642924Disabling compacts and flushes for region at 1676290642924Disabling writes for close at 1676290642925 (+1 ms)Obtaining lock to block concurrent updates at 1676290642925Preparing flush snapshotting stores in 601e1d5b9375e4ae31702b41106955a2 at 1676290642925Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676290642925Flushing stores of testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. at 1676290642925Flushing 601e1d5b9375e4ae31702b41106955a2/cf1: creating writer at 1676290642927 (+2 ms)Flushing 601e1d5b9375e4ae31702b41106955a2/cf1: appending metadata at 1676290642930 (+3 ms)Flushing 601e1d5b9375e4ae31702b41106955a2/cf1: closing flushed file at 1676290642930Flushing 601e1d5b9375e4ae31702b41106955a2/cf2: creating writer at 1676290642940 (+10 ms)Flushing 601e1d5b9375e4ae31702b41106955a2/cf2: appending metadata at 1676290642942 (+2 ms)Flushing 601e1d5b9375e4ae31702b41106955a2/cf2: closing flushed file at 1676290642942Flushing 601e1d5b9375e4ae31702b41106955a2/cf1: reopening flushed file at 1676290642957 (+15 ms)Flushing 601e1d5b9375e4ae31702b41106955a2/cf2: reopening flushed file at 1676290642962 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 601e1d5b9375e4ae31702b41106955a2 in 41ms, sequenceid=82, compaction requested=true at 1676290642966 (+4 ms)Writing region close event to WAL at 1676290642967 (+1 ms)Running coprocessor post-close hooks at 1676290642970 (+3 ms)Closed at 1676290642971 (+1 ms) 2023-02-13 12:17:22,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,973 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=601e1d5b9375e4ae31702b41106955a2, regionState=CLOSED 2023-02-13 12:17:22,973 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290642973"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290642973"}]},"ts":"1676290642973"} 2023-02-13 12:17:22,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-13 12:17:22,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 601e1d5b9375e4ae31702b41106955a2, server=jenkins-hbase16.apache.org,33055,1676290533708 in 201 msec 2023-02-13 12:17:22,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-13 12:17:22,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=601e1d5b9375e4ae31702b41106955a2, UNASSIGN in 204 msec 2023-02-13 12:17:22,985 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=601e1d5b9375e4ae31702b41106955a2, threads=14 2023-02-13 12:17:22,985 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/0eac841ef3904705a610ef72d9aa87b7 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,985 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/1e4f826918c0404faf02976d4b614d2c for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,985 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/3d3d46435f4f47edb556d730becdc676 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,985 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/9ea3dcb194954417a5194c616b9a77e8 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,985 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/bc6eee85e4714e168ee9c8d17c42782c for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,985 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/dfa08e52d7b44a4393da3df1bee03ea3 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,986 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/026c94d1e64c429ebf09713989f9af36 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,986 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/ecafb8b2189b4e04a9c161d5db2162f6 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,988 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/198dab6d84724bc1965f0118b668b48f for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,990 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/1fb43537a5264935b328ccb20628a27e for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,990 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/4a1f71a232c24bb49de6e98e8f879cff for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,990 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/c48077b53ddb4b89b5911e872a851b79 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,992 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/cb3984de5d044e76869ea720632959e3 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,992 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/da243e4d8bfb4ca2a4b48cf2ac903455 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:22,994 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/0eac841ef3904705a610ef72d9aa87b7, top=false 2023-02-13 12:17:22,995 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/3d3d46435f4f47edb556d730becdc676, top=false 2023-02-13 12:17:22,995 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/198dab6d84724bc1965f0118b668b48f, top=false 2023-02-13 12:17:23,002 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/1e4f826918c0404faf02976d4b614d2c, top=false 2023-02-13 12:17:23,004 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/bc6eee85e4714e168ee9c8d17c42782c, top=false 2023-02-13 12:17:23,005 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/4a1f71a232c24bb49de6e98e8f879cff, top=false 2023-02-13 12:17:23,006 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/cb3984de5d044e76869ea720632959e3, top=false 2023-02-13 12:17:23,006 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/9ea3dcb194954417a5194c616b9a77e8, top=false 2023-02-13 12:17:23,006 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/dfa08e52d7b44a4393da3df1bee03ea3, top=false 2023-02-13 12:17:23,006 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/ecafb8b2189b4e04a9c161d5db2162f6, top=false 2023-02-13 12:17:23,007 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/1fb43537a5264935b328ccb20628a27e, top=false 2023-02-13 12:17:23,008 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/026c94d1e64c429ebf09713989f9af36, top=false 2023-02-13 12:17:23,009 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/da243e4d8bfb4ca2a4b48cf2ac903455, top=false 2023-02-13 12:17:23,010 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/c48077b53ddb4b89b5911e872a851b79, top=false 2023-02-13 12:17:23,011 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-0eac841ef3904705a610ef72d9aa87b7 for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,011 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-3d3d46435f4f47edb556d730becdc676 for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,021 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/3d3d46435f4f47edb556d730becdc676 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,021 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/0eac841ef3904705a610ef72d9aa87b7 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,024 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-dfa08e52d7b44a4393da3df1bee03ea3 for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,024 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-198dab6d84724bc1965f0118b668b48f for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,025 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-ecafb8b2189b4e04a9c161d5db2162f6 for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,025 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1e4f826918c0404faf02976d4b614d2c for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,055 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/dfa08e52d7b44a4393da3df1bee03ea3 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,056 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/ecafb8b2189b4e04a9c161d5db2162f6 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,057 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/1e4f826918c0404faf02976d4b614d2c for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,057 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/198dab6d84724bc1965f0118b668b48f for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,060 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-4a1f71a232c24bb49de6e98e8f879cff for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,060 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-bc6eee85e4714e168ee9c8d17c42782c for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,060 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1fb43537a5264935b328ccb20628a27e for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,060 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-cb3984de5d044e76869ea720632959e3 for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,064 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-c48077b53ddb4b89b5911e872a851b79 for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,065 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-9ea3dcb194954417a5194c616b9a77e8 for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,065 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-026c94d1e64c429ebf09713989f9af36 for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,065 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/.splits/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-da243e4d8bfb4ca2a4b48cf2ac903455 for child: 52d2efcd0f602bb990ec308869fa8e25, parent: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,069 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/bc6eee85e4714e168ee9c8d17c42782c for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,069 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/da243e4d8bfb4ca2a4b48cf2ac903455 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,069 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/c48077b53ddb4b89b5911e872a851b79 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,069 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/4a1f71a232c24bb49de6e98e8f879cff for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,069 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/1fb43537a5264935b328ccb20628a27e for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,070 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/cb3984de5d044e76869ea720632959e3 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,071 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/9ea3dcb194954417a5194c616b9a77e8 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,071 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/026c94d1e64c429ebf09713989f9af36 for region: 601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,072 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 601e1d5b9375e4ae31702b41106955a2 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-13 12:17:23,105 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:17:23,106 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:17:23,107 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676290643107"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676290643107"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676290643107"}]},"ts":"1676290643107"} 2023-02-13 12:17:23,107 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676290643107"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290643107"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290643107"}]},"ts":"1676290643107"} 2023-02-13 12:17:23,108 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676290643107"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290643107"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290643107"}]},"ts":"1676290643107"} 2023-02-13 12:17:23,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52d2efcd0f602bb990ec308869fa8e25, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7f145ae641beeb882ac3c2e2344ae0f0, ASSIGN}] 2023-02-13 12:17:23,111 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7f145ae641beeb882ac3c2e2344ae0f0, ASSIGN 2023-02-13 12:17:23,112 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7f145ae641beeb882ac3c2e2344ae0f0, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:17:23,113 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52d2efcd0f602bb990ec308869fa8e25, ASSIGN 2023-02-13 12:17:23,113 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52d2efcd0f602bb990ec308869fa8e25, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,33055,1676290533708; forceNewPlan=false, retain=false 2023-02-13 12:17:23,262 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 12:17:23,264 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=7f145ae641beeb882ac3c2e2344ae0f0, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:23,264 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676290643264"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290643264"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290643264"}]},"ts":"1676290643264"} 2023-02-13 12:17:23,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 7f145ae641beeb882ac3c2e2344ae0f0, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:23,268 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=52d2efcd0f602bb990ec308869fa8e25, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:23,268 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676290643268"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290643268"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290643268"}]},"ts":"1676290643268"} 2023-02-13 12:17:23,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 52d2efcd0f602bb990ec308869fa8e25, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:23,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25. 2023-02-13 12:17:23,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 52d2efcd0f602bb990ec308869fa8e25, NAME => 'testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.', STARTKEY => '', ENDKEY => '71'} 2023-02-13 12:17:23,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:17:23,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:23,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,422 INFO [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,423 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1 2023-02-13 12:17:23,423 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1 2023-02-13 12:17:23,423 INFO [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 52d2efcd0f602bb990ec308869fa8e25 columnFamilyName cf1 2023-02-13 12:17:23,427 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-026c94d1e64c429ebf09713989f9af36 2023-02-13 12:17:23,430 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-198dab6d84724bc1965f0118b668b48f 2023-02-13 12:17:23,433 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1fb43537a5264935b328ccb20628a27e 2023-02-13 12:17:23,436 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-4a1f71a232c24bb49de6e98e8f879cff 2023-02-13 12:17:23,439 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-c48077b53ddb4b89b5911e872a851b79 2023-02-13 12:17:23,441 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-cb3984de5d044e76869ea720632959e3 2023-02-13 12:17:23,444 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-da243e4d8bfb4ca2a4b48cf2ac903455 2023-02-13 12:17:23,444 INFO [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(310): Store=52d2efcd0f602bb990ec308869fa8e25/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:23,444 INFO [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,445 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2 2023-02-13 12:17:23,445 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2 2023-02-13 12:17:23,446 INFO [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 52d2efcd0f602bb990ec308869fa8e25 columnFamilyName cf2 2023-02-13 12:17:23,450 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-0eac841ef3904705a610ef72d9aa87b7 2023-02-13 12:17:23,453 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1e4f826918c0404faf02976d4b614d2c 2023-02-13 12:17:23,455 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-3d3d46435f4f47edb556d730becdc676 2023-02-13 12:17:23,457 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-9ea3dcb194954417a5194c616b9a77e8 2023-02-13 12:17:23,460 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-bc6eee85e4714e168ee9c8d17c42782c 2023-02-13 12:17:23,463 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-dfa08e52d7b44a4393da3df1bee03ea3 2023-02-13 12:17:23,465 DEBUG [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-ecafb8b2189b4e04a9c161d5db2162f6 2023-02-13 12:17:23,466 INFO [StoreOpener-52d2efcd0f602bb990ec308869fa8e25-1] regionserver.HStore(310): Store=52d2efcd0f602bb990ec308869fa8e25/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:23,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:17:23,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 52d2efcd0f602bb990ec308869fa8e25; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64991500, jitterRate=-0.03155118227005005}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:17:23,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 52d2efcd0f602bb990ec308869fa8e25: Running coprocessor pre-open hook at 1676290643421Writing region info on filesystem at 1676290643421Initializing all the Stores at 1676290643421Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290643422 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290643422Cleaning up temporary data from old regions at 1676290643468 (+46 ms)Cleaning up detritus from prior splits at 1676290643468Running coprocessor post-open hooks at 1676290643470 (+2 ms)Region opened successfully at 1676290643470 2023-02-13 12:17:23,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25., pid=134, masterSystemTime=1676290643417 2023-02-13 12:17:23,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:23,471 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:17:23,471 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:17:23,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:17:23,471 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 12:17:23,472 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 12:17:23,472 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1912): 52d2efcd0f602bb990ec308869fa8e25/cf1 is initiating minor compaction (all files) 2023-02-13 12:17:23,472 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1912): 52d2efcd0f602bb990ec308869fa8e25/cf2 is initiating minor compaction (all files) 2023-02-13 12:17:23,472 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 52d2efcd0f602bb990ec308869fa8e25/cf1 in testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25. 2023-02-13 12:17:23,472 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 52d2efcd0f602bb990ec308869fa8e25/cf2 in testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25. 2023-02-13 12:17:23,472 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-cb3984de5d044e76869ea720632959e3, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-198dab6d84724bc1965f0118b668b48f, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-026c94d1e64c429ebf09713989f9af36, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1fb43537a5264935b328ccb20628a27e, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-c48077b53ddb4b89b5911e872a851b79, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-4a1f71a232c24bb49de6e98e8f879cff, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-da243e4d8bfb4ca2a4b48cf2ac903455] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/.tmp, totalSize=34.9 K 2023-02-13 12:17:23,472 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-9ea3dcb194954417a5194c616b9a77e8, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-3d3d46435f4f47edb556d730becdc676, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-ecafb8b2189b4e04a9c161d5db2162f6, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-dfa08e52d7b44a4393da3df1bee03ea3, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-bc6eee85e4714e168ee9c8d17c42782c, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-0eac841ef3904705a610ef72d9aa87b7, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1e4f826918c0404faf02976d4b614d2c] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/.tmp, totalSize=34.9 K 2023-02-13 12:17:23,472 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-cb3984de5d044e76869ea720632959e3, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290641410 2023-02-13 12:17:23,472 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-9ea3dcb194954417a5194c616b9a77e8, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290641410 2023-02-13 12:17:23,473 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-198dab6d84724bc1965f0118b668b48f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290641632 2023-02-13 12:17:23,473 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-3d3d46435f4f47edb556d730becdc676, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290641632 2023-02-13 12:17:23,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25. 2023-02-13 12:17:23,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25. 2023-02-13 12:17:23,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0. 2023-02-13 12:17:23,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7f145ae641beeb882ac3c2e2344ae0f0, NAME => 'testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0.', STARTKEY => '71', ENDKEY => ''} 2023-02-13 12:17:23,473 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-026c94d1e64c429ebf09713989f9af36, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290641850 2023-02-13 12:17:23,473 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-ecafb8b2189b4e04a9c161d5db2162f6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290641850 2023-02-13 12:17:23,473 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=52d2efcd0f602bb990ec308869fa8e25, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:23,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:17:23,473 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1fb43537a5264935b328ccb20628a27e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290642067 2023-02-13 12:17:23,474 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676290643473"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290643473"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290643473"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290643473"}]},"ts":"1676290643473"} 2023-02-13 12:17:23,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:23,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,474 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-dfa08e52d7b44a4393da3df1bee03ea3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290642067 2023-02-13 12:17:23,474 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-c48077b53ddb4b89b5911e872a851b79, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290642342 2023-02-13 12:17:23,475 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-bc6eee85e4714e168ee9c8d17c42782c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290642342 2023-02-13 12:17:23,475 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-4a1f71a232c24bb49de6e98e8f879cff, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290642552 2023-02-13 12:17:23,475 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-0eac841ef3904705a610ef72d9aa87b7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290642552 2023-02-13 12:17:23,475 INFO [StoreOpener-7f145ae641beeb882ac3c2e2344ae0f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,475 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-da243e4d8bfb4ca2a4b48cf2ac903455, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290642761 2023-02-13 12:17:23,476 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1e4f826918c0404faf02976d4b614d2c, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290642761 2023-02-13 12:17:23,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-13 12:17:23,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 52d2efcd0f602bb990ec308869fa8e25, server=jenkins-hbase16.apache.org,33055,1676290533708 in 206 msec 2023-02-13 12:17:23,478 DEBUG [StoreOpener-7f145ae641beeb882ac3c2e2344ae0f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/cf1 2023-02-13 12:17:23,478 DEBUG [StoreOpener-7f145ae641beeb882ac3c2e2344ae0f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/cf1 2023-02-13 12:17:23,478 INFO [StoreOpener-7f145ae641beeb882ac3c2e2344ae0f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7f145ae641beeb882ac3c2e2344ae0f0 columnFamilyName cf1 2023-02-13 12:17:23,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52d2efcd0f602bb990ec308869fa8e25, ASSIGN in 367 msec 2023-02-13 12:17:23,479 INFO [StoreOpener-7f145ae641beeb882ac3c2e2344ae0f0-1] regionserver.HStore(310): Store=7f145ae641beeb882ac3c2e2344ae0f0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:23,479 INFO [StoreOpener-7f145ae641beeb882ac3c2e2344ae0f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,480 DEBUG [StoreOpener-7f145ae641beeb882ac3c2e2344ae0f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/cf2 2023-02-13 12:17:23,480 DEBUG [StoreOpener-7f145ae641beeb882ac3c2e2344ae0f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/cf2 2023-02-13 12:17:23,480 INFO [StoreOpener-7f145ae641beeb882ac3c2e2344ae0f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7f145ae641beeb882ac3c2e2344ae0f0 columnFamilyName cf2 2023-02-13 12:17:23,481 INFO [StoreOpener-7f145ae641beeb882ac3c2e2344ae0f0-1] regionserver.HStore(310): Store=7f145ae641beeb882ac3c2e2344ae0f0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:23,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:17:23,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 7f145ae641beeb882ac3c2e2344ae0f0; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63363690, jitterRate=-0.0558074414730072}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:17:23,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 7f145ae641beeb882ac3c2e2344ae0f0: Running coprocessor pre-open hook at 1676290643474Writing region info on filesystem at 1676290643474Initializing all the Stores at 1676290643475 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290643475Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290643475Cleaning up temporary data from old regions at 1676290643483 (+8 ms)Cleaning up detritus from prior splits at 1676290643483Running coprocessor post-open hooks at 1676290643486 (+3 ms)Region opened successfully at 1676290643486 2023-02-13 12:17:23,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0., pid=133, masterSystemTime=1676290643417 2023-02-13 12:17:23,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0. 2023-02-13 12:17:23,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0. 2023-02-13 12:17:23,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=7f145ae641beeb882ac3c2e2344ae0f0, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:23,489 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676290643489"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290643489"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290643489"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290643489"}]},"ts":"1676290643489"} 2023-02-13 12:17:23,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-13 12:17:23,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 7f145ae641beeb882ac3c2e2344ae0f0, server=jenkins-hbase16.apache.org,33055,1676290533708 in 224 msec 2023-02-13 12:17:23,494 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] throttle.PressureAwareThroughputController(145): 52d2efcd0f602bb990ec308869fa8e25#cf2#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:17:23,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-13 12:17:23,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7f145ae641beeb882ac3c2e2344ae0f0, ASSIGN in 383 msec 2023-02-13 12:17:23,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=601e1d5b9375e4ae31702b41106955a2, daughterA=52d2efcd0f602bb990ec308869fa8e25, daughterB=7f145ae641beeb882ac3c2e2344ae0f0 in 727 msec 2023-02-13 12:17:23,504 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] throttle.PressureAwareThroughputController(145): 52d2efcd0f602bb990ec308869fa8e25#cf1#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:17:23,515 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/.tmp/cf2/d92499da2ee64f538528ae81442be299 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/d92499da2ee64f538528ae81442be299 2023-02-13 12:17:23,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33055] ipc.CallRunner(144): callId: 991 service: ClientService methodName: Scan size: 157 connection: 188.40.62.62:45960 deadline: 1676290703519, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2. is not online on jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:23,520 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 52d2efcd0f602bb990ec308869fa8e25/cf2 of 52d2efcd0f602bb990ec308869fa8e25 into d92499da2ee64f538528ae81442be299(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:17:23,520 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 52d2efcd0f602bb990ec308869fa8e25: 2023-02-13 12:17:23,521 INFO [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25., storeName=52d2efcd0f602bb990ec308869fa8e25/cf2, priority=9, startTime=1676290643471; duration=0sec 2023-02-13 12:17:23,521 DEBUG [RS:0;jenkins-hbase16:33055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:23,523 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/.tmp/cf1/182d45e1478a4e16a585a441473f132e as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/182d45e1478a4e16a585a441473f132e 2023-02-13 12:17:23,527 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 52d2efcd0f602bb990ec308869fa8e25/cf1 of 52d2efcd0f602bb990ec308869fa8e25 into 182d45e1478a4e16a585a441473f132e(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:17:23,527 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 52d2efcd0f602bb990ec308869fa8e25: 2023-02-13 12:17:23,527 INFO [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25., storeName=52d2efcd0f602bb990ec308869fa8e25/cf1, priority=9, startTime=1676290643471; duration=0sec 2023-02-13 12:17:23,528 DEBUG [RS:0;jenkins-hbase16:33055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:23,629 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-13 12:17:23,633 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:17:23,634 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-13 12:17:23,635 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitTableRegionEmptyDaughter 2023-02-13 12:17:23,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 12:17:23,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 12:17:23,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290643638"}]},"ts":"1676290643638"} 2023-02-13 12:17:23,639 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-13 12:17:23,675 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-13 12:17:23,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52d2efcd0f602bb990ec308869fa8e25, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7f145ae641beeb882ac3c2e2344ae0f0, UNASSIGN}] 2023-02-13 12:17:23,676 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7f145ae641beeb882ac3c2e2344ae0f0, UNASSIGN 2023-02-13 12:17:23,677 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=7f145ae641beeb882ac3c2e2344ae0f0, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:23,677 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676290643677"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290643677"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290643677"}]},"ts":"1676290643677"} 2023-02-13 12:17:23,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 7f145ae641beeb882ac3c2e2344ae0f0, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:23,680 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52d2efcd0f602bb990ec308869fa8e25, UNASSIGN 2023-02-13 12:17:23,680 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=52d2efcd0f602bb990ec308869fa8e25, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:23,680 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676290643680"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290643680"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290643680"}]},"ts":"1676290643680"} 2023-02-13 12:17:23,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 52d2efcd0f602bb990ec308869fa8e25, server=jenkins-hbase16.apache.org,33055,1676290533708}] 2023-02-13 12:17:23,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 12:17:23,829 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 52d2efcd0f602bb990ec308869fa8e25, disabling compactions & flushes 2023-02-13 12:17:23,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25. 2023-02-13 12:17:23,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25. 2023-02-13 12:17:23,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25. after waiting 0 ms 2023-02-13 12:17:23,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25. 2023-02-13 12:17:23,830 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-cb3984de5d044e76869ea720632959e3, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-198dab6d84724bc1965f0118b668b48f, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-026c94d1e64c429ebf09713989f9af36, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1fb43537a5264935b328ccb20628a27e, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-c48077b53ddb4b89b5911e872a851b79, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-4a1f71a232c24bb49de6e98e8f879cff, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-da243e4d8bfb4ca2a4b48cf2ac903455] to archive 2023-02-13 12:17:23,831 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:17:23,833 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-cb3984de5d044e76869ea720632959e3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-cb3984de5d044e76869ea720632959e3 2023-02-13 12:17:23,834 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-198dab6d84724bc1965f0118b668b48f to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-198dab6d84724bc1965f0118b668b48f 2023-02-13 12:17:23,835 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-026c94d1e64c429ebf09713989f9af36 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-026c94d1e64c429ebf09713989f9af36 2023-02-13 12:17:23,836 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1fb43537a5264935b328ccb20628a27e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1fb43537a5264935b328ccb20628a27e 2023-02-13 12:17:23,837 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-c48077b53ddb4b89b5911e872a851b79 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-c48077b53ddb4b89b5911e872a851b79 2023-02-13 12:17:23,838 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-4a1f71a232c24bb49de6e98e8f879cff to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-4a1f71a232c24bb49de6e98e8f879cff 2023-02-13 12:17:23,838 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-da243e4d8bfb4ca2a4b48cf2ac903455 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-da243e4d8bfb4ca2a4b48cf2ac903455 2023-02-13 12:17:23,840 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-9ea3dcb194954417a5194c616b9a77e8, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-3d3d46435f4f47edb556d730becdc676, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-ecafb8b2189b4e04a9c161d5db2162f6, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-dfa08e52d7b44a4393da3df1bee03ea3, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-bc6eee85e4714e168ee9c8d17c42782c, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-0eac841ef3904705a610ef72d9aa87b7, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1e4f826918c0404faf02976d4b614d2c] to archive 2023-02-13 12:17:23,840 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:17:23,841 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-9ea3dcb194954417a5194c616b9a77e8 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-9ea3dcb194954417a5194c616b9a77e8 2023-02-13 12:17:23,842 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-3d3d46435f4f47edb556d730becdc676 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-3d3d46435f4f47edb556d730becdc676 2023-02-13 12:17:23,843 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-ecafb8b2189b4e04a9c161d5db2162f6 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-ecafb8b2189b4e04a9c161d5db2162f6 2023-02-13 12:17:23,844 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-dfa08e52d7b44a4393da3df1bee03ea3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-dfa08e52d7b44a4393da3df1bee03ea3 2023-02-13 12:17:23,845 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-bc6eee85e4714e168ee9c8d17c42782c to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-bc6eee85e4714e168ee9c8d17c42782c 2023-02-13 12:17:23,846 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-0eac841ef3904705a610ef72d9aa87b7 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-0eac841ef3904705a610ef72d9aa87b7 2023-02-13 12:17:23,847 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1e4f826918c0404faf02976d4b614d2c to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/testSplitTableRegionEmptyDaughter=601e1d5b9375e4ae31702b41106955a2-1e4f826918c0404faf02976d4b614d2c 2023-02-13 12:17:23,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 12:17:23,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:23,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25. 2023-02-13 12:17:23,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 52d2efcd0f602bb990ec308869fa8e25: Waiting for close lock at 1676290643830Running coprocessor pre-close hooks at 1676290643830Disabling compacts and flushes for region at 1676290643830Disabling writes for close at 1676290643830Writing region close event to WAL at 1676290643847 (+17 ms)Running coprocessor post-close hooks at 1676290643850 (+3 ms)Closed at 1676290643850 2023-02-13 12:17:23,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 7f145ae641beeb882ac3c2e2344ae0f0, disabling compactions & flushes 2023-02-13 12:17:23,853 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0. 2023-02-13 12:17:23,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0. 2023-02-13 12:17:23,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0. after waiting 0 ms 2023-02-13 12:17:23,853 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=52d2efcd0f602bb990ec308869fa8e25, regionState=CLOSED 2023-02-13 12:17:23,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0. 2023-02-13 12:17:23,853 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676290643853"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290643853"}]},"ts":"1676290643853"} 2023-02-13 12:17:23,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-13 12:17:23,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 52d2efcd0f602bb990ec308869fa8e25, server=jenkins-hbase16.apache.org,33055,1676290533708 in 173 msec 2023-02-13 12:17:23,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-13 12:17:23,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:23,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52d2efcd0f602bb990ec308869fa8e25, UNASSIGN in 179 msec 2023-02-13 12:17:23,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0. 2023-02-13 12:17:23,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 7f145ae641beeb882ac3c2e2344ae0f0: Waiting for close lock at 1676290643852Running coprocessor pre-close hooks at 1676290643852Disabling compacts and flushes for region at 1676290643852Disabling writes for close at 1676290643853 (+1 ms)Writing region close event to WAL at 1676290643853Running coprocessor post-close hooks at 1676290643856 (+3 ms)Closed at 1676290643856 2023-02-13 12:17:23,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,858 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=7f145ae641beeb882ac3c2e2344ae0f0, regionState=CLOSED 2023-02-13 12:17:23,858 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676290643857"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290643857"}]},"ts":"1676290643857"} 2023-02-13 12:17:23,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-13 12:17:23,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 7f145ae641beeb882ac3c2e2344ae0f0, server=jenkins-hbase16.apache.org,33055,1676290533708 in 181 msec 2023-02-13 12:17:23,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-13 12:17:23,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7f145ae641beeb882ac3c2e2344ae0f0, UNASSIGN in 184 msec 2023-02-13 12:17:23,862 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290643862"}]},"ts":"1676290643862"} 2023-02-13 12:17:23,863 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-13 12:17:23,866 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-13 12:17:23,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 232 msec 2023-02-13 12:17:23,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 12:17:23,941 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-13 12:17:23,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitTableRegionEmptyDaughter 2023-02-13 12:17:23,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 12:17:23,944 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 12:17:23,945 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 12:17:23,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 12:17:23,949 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,949 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,949 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,951 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/recovered.edits] 2023-02-13 12:17:23,951 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/recovered.edits] 2023-02-13 12:17:23,951 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/recovered.edits] 2023-02-13 12:17:23,953 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/182d45e1478a4e16a585a441473f132e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf1/182d45e1478a4e16a585a441473f132e 2023-02-13 12:17:23,953 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/026c94d1e64c429ebf09713989f9af36 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/026c94d1e64c429ebf09713989f9af36 2023-02-13 12:17:23,955 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/198dab6d84724bc1965f0118b668b48f to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/198dab6d84724bc1965f0118b668b48f 2023-02-13 12:17:23,955 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/recovered.edits/88.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0/recovered.edits/88.seqid 2023-02-13 12:17:23,955 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/d92499da2ee64f538528ae81442be299 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/cf2/d92499da2ee64f538528ae81442be299 2023-02-13 12:17:23,956 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/7f145ae641beeb882ac3c2e2344ae0f0 2023-02-13 12:17:23,956 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/1fb43537a5264935b328ccb20628a27e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/1fb43537a5264935b328ccb20628a27e 2023-02-13 12:17:23,957 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/4a1f71a232c24bb49de6e98e8f879cff to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/4a1f71a232c24bb49de6e98e8f879cff 2023-02-13 12:17:23,958 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/recovered.edits/90.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25/recovered.edits/90.seqid 2023-02-13 12:17:23,958 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/c48077b53ddb4b89b5911e872a851b79 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/c48077b53ddb4b89b5911e872a851b79 2023-02-13 12:17:23,959 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/52d2efcd0f602bb990ec308869fa8e25 2023-02-13 12:17:23,960 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/cb3984de5d044e76869ea720632959e3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/cb3984de5d044e76869ea720632959e3 2023-02-13 12:17:23,961 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/da243e4d8bfb4ca2a4b48cf2ac903455 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf1/da243e4d8bfb4ca2a4b48cf2ac903455 2023-02-13 12:17:23,964 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/0eac841ef3904705a610ef72d9aa87b7 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/0eac841ef3904705a610ef72d9aa87b7 2023-02-13 12:17:23,965 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/1e4f826918c0404faf02976d4b614d2c to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/1e4f826918c0404faf02976d4b614d2c 2023-02-13 12:17:23,966 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/3d3d46435f4f47edb556d730becdc676 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/3d3d46435f4f47edb556d730becdc676 2023-02-13 12:17:23,967 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/9ea3dcb194954417a5194c616b9a77e8 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/9ea3dcb194954417a5194c616b9a77e8 2023-02-13 12:17:23,968 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/bc6eee85e4714e168ee9c8d17c42782c to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/bc6eee85e4714e168ee9c8d17c42782c 2023-02-13 12:17:23,969 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/dfa08e52d7b44a4393da3df1bee03ea3 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/dfa08e52d7b44a4393da3df1bee03ea3 2023-02-13 12:17:23,970 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/ecafb8b2189b4e04a9c161d5db2162f6 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/cf2/ecafb8b2189b4e04a9c161d5db2162f6 2023-02-13 12:17:23,972 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/recovered.edits/85.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2/recovered.edits/85.seqid 2023-02-13 12:17:23,972 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegionEmptyDaughter/601e1d5b9375e4ae31702b41106955a2 2023-02-13 12:17:23,972 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 12:17:23,974 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 12:17:23,976 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-13 12:17:23,978 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-13 12:17:23,979 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 12:17:23,979 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-13 12:17:23,979 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290643979"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:23,979 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290643979"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:23,979 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290643979"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:23,980 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 12:17:23,980 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 601e1d5b9375e4ae31702b41106955a2, NAME => 'testSplitTableRegionEmptyDaughter,,1676290640906.601e1d5b9375e4ae31702b41106955a2.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 52d2efcd0f602bb990ec308869fa8e25, NAME => 'testSplitTableRegionEmptyDaughter,,1676290642767.52d2efcd0f602bb990ec308869fa8e25.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 7f145ae641beeb882ac3c2e2344ae0f0, NAME => 'testSplitTableRegionEmptyDaughter,71,1676290642767.7f145ae641beeb882ac3c2e2344ae0f0.', STARTKEY => '71', ENDKEY => ''}] 2023-02-13 12:17:23,980 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-13 12:17:23,980 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676290643980"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:23,982 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-13 12:17:23,992 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 12:17:23,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 50 msec 2023-02-13 12:17:24,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 12:17:24,047 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-13 12:17:24,059 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376 (was 377), OpenFileDescriptor=839 (was 841), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=247 (was 247), ProcessCount=167 (was 167), AvailableMemoryMB=5358 (was 5406) 2023-02-13 12:17:24,071 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376, OpenFileDescriptor=839, MaxFileDescriptor=60000, SystemLoadAverage=247, ProcessCount=167, AvailableMemoryMB=5358 2023-02-13 12:17:24,072 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:17:24,083 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 12:17:24,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 12:17:24,085 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-13 12:17:24,087 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:17:24,087 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:17:24,088 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:24,089 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659 empty. 2023-02-13 12:17:24,089 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:24,089 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 12:17:24,097 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-13 12:17:24,098 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => e0497e21f354e69dc75ccda3c9bde659, NAME => 'testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp 2023-02-13 12:17:24,108 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:24,108 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing e0497e21f354e69dc75ccda3c9bde659, disabling compactions & flushes 2023-02-13 12:17:24,108 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,108 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,108 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. after waiting 0 ms 2023-02-13 12:17:24,109 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,109 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,109 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for e0497e21f354e69dc75ccda3c9bde659: Waiting for close lock at 1676290644108Disabling compacts and flushes for region at 1676290644108Disabling writes for close at 1676290644109 (+1 ms)Writing region close event to WAL at 1676290644109Closed at 1676290644109 2023-02-13 12:17:24,110 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:17:24,111 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290644111"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290644111"}]},"ts":"1676290644111"} 2023-02-13 12:17:24,112 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:17:24,113 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:17:24,113 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290644113"}]},"ts":"1676290644113"} 2023-02-13 12:17:24,114 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-13 12:17:24,133 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 12:17:24,134 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 12:17:24,134 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 12:17:24,134 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 12:17:24,134 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 12:17:24,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e0497e21f354e69dc75ccda3c9bde659, ASSIGN}] 2023-02-13 12:17:24,136 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e0497e21f354e69dc75ccda3c9bde659, ASSIGN 2023-02-13 12:17:24,136 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=e0497e21f354e69dc75ccda3c9bde659, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,33961,1676290533759; forceNewPlan=false, retain=false 2023-02-13 12:17:24,286 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 12:17:24,287 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=e0497e21f354e69dc75ccda3c9bde659, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,287 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290644287"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290644287"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290644287"}]},"ts":"1676290644287"} 2023-02-13 12:17:24,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure e0497e21f354e69dc75ccda3c9bde659, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:17:24,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e0497e21f354e69dc75ccda3c9bde659, NAME => 'testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:17:24,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:17:24,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:24,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:24,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:24,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:24,447 INFO [StoreOpener-e0497e21f354e69dc75ccda3c9bde659-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:24,448 DEBUG [StoreOpener-e0497e21f354e69dc75ccda3c9bde659-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1 2023-02-13 12:17:24,448 DEBUG [StoreOpener-e0497e21f354e69dc75ccda3c9bde659-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1 2023-02-13 12:17:24,449 INFO [StoreOpener-e0497e21f354e69dc75ccda3c9bde659-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e0497e21f354e69dc75ccda3c9bde659 columnFamilyName cf1 2023-02-13 12:17:24,449 INFO [StoreOpener-e0497e21f354e69dc75ccda3c9bde659-1] regionserver.HStore(310): Store=e0497e21f354e69dc75ccda3c9bde659/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:24,449 INFO [StoreOpener-e0497e21f354e69dc75ccda3c9bde659-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:24,450 DEBUG [StoreOpener-e0497e21f354e69dc75ccda3c9bde659-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2 2023-02-13 12:17:24,450 DEBUG [StoreOpener-e0497e21f354e69dc75ccda3c9bde659-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2 2023-02-13 12:17:24,450 INFO [StoreOpener-e0497e21f354e69dc75ccda3c9bde659-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e0497e21f354e69dc75ccda3c9bde659 columnFamilyName cf2 2023-02-13 12:17:24,451 INFO [StoreOpener-e0497e21f354e69dc75ccda3c9bde659-1] regionserver.HStore(310): Store=e0497e21f354e69dc75ccda3c9bde659/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:24,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:24,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:24,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:17:24,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:24,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:17:24,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened e0497e21f354e69dc75ccda3c9bde659; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61260506, jitterRate=-0.08714732527732849}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:17:24,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for e0497e21f354e69dc75ccda3c9bde659: Running coprocessor pre-open hook at 1676290644446Writing region info on filesystem at 1676290644446Initializing all the Stores at 1676290644447 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290644447Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290644447Cleaning up temporary data from old regions at 1676290644452 (+5 ms)Cleaning up detritus from prior splits at 1676290644453 (+1 ms)Running coprocessor post-open hooks at 1676290644455 (+2 ms)Region opened successfully at 1676290644455 2023-02-13 12:17:24,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659., pid=143, masterSystemTime=1676290644440 2023-02-13 12:17:24,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=e0497e21f354e69dc75ccda3c9bde659, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,458 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290644458"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290644458"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290644458"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290644458"}]},"ts":"1676290644458"} 2023-02-13 12:17:24,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-13 12:17:24,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure e0497e21f354e69dc75ccda3c9bde659, server=jenkins-hbase16.apache.org,33961,1676290533759 in 170 msec 2023-02-13 12:17:24,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-13 12:17:24,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e0497e21f354e69dc75ccda3c9bde659, ASSIGN in 326 msec 2023-02-13 12:17:24,462 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:17:24,462 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290644462"}]},"ts":"1676290644462"} 2023-02-13 12:17:24,463 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-13 12:17:24,475 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:17:24,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 391 msec 2023-02-13 12:17:24,592 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:24,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 12:17:24,594 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 12:17:24,594 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:24,594 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 12:17:24,594 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 12:17:24,594 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,594 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,600 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,600 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,600 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,600 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,600 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,600 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:24,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:24,601 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:24,601 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:24,601 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:24,601 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:24,602 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:24,601 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:24,602 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:24,602 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:24,602 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:24,602 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,602 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:24,602 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:24,602 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:24,602 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:24,602 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. started... 2023-02-13 12:17:24,602 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:24,603 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0497e21f354e69dc75ccda3c9bde659 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 12:17:24,608 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,611 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/e8f15f3829c64a0caad4220fa3303a80 2023-02-13 12:17:24,616 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,616 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,616 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:24,616 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,617 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:24,624 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/f8c192c06cac430ba4b13bf714ad2580 2023-02-13 12:17:24,627 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/e8f15f3829c64a0caad4220fa3303a80 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80 2023-02-13 12:17:24,630 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:17:24,631 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/f8c192c06cac430ba4b13bf714ad2580 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580 2023-02-13 12:17:24,634 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:17:24,635 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e0497e21f354e69dc75ccda3c9bde659 in 32ms, sequenceid=6, compaction requested=false 2023-02-13 12:17:24,635 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-13 12:17:24,635 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0497e21f354e69dc75ccda3c9bde659: 2023-02-13 12:17:24,635 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,635 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:24,636 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:24,636 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,636 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:24,636 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:24,641 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,641 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:24,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:24,642 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,642 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:24,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:24,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:24,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:24,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 12:17:24,643 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 12:17:24,643 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@265339cb[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:24,667 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,667 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,667 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,667 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,667 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:24,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,667 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:24,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,667 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:24,667 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:24,668 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:24,668 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:24,668 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:24,668 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:24,668 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:24,668 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:24,682 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:24,682 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:24,682 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:24,691 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:24,691 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,691 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:24,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:24,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:24,691 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:24,692 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:24,692 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:24,692 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:24,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:24,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:17:24,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 12:17:24,695 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 12:17:24,695 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:24,695 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:24,695 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 12:17:24,695 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:24,700 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,700 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,700 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:24,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,701 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:24,701 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:24,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,702 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:24,703 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,703 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,708 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,708 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,708 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,708 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,708 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,708 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 12:17:24,708 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:24,708 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,708 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,708 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,708 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:24,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:24,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,709 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,731 ERROR [BP-591480327-188.40.62.62-1676290528441 heartbeating to localhost.localdomain/127.0.0.1:44273] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-591480327-188.40.62.62-1676290528441 (Datanode Uuid 14b0d84a-bcac-40f1-8e55-f251954dc488) service to localhost.localdomain/127.0.0.1:44273 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 12:17:24,809 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:24,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:24,819 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:24,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:24,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 12:17:24,821 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 12:17:24,821 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:24,821 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 12:17:24,821 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 12:17:24,821 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,821 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,825 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,825 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,825 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,825 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,825 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,825 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:24,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:24,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:24,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:24,826 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:24,826 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:24,826 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:24,826 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:24,827 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:24,827 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:24,827 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:24,827 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:24,827 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:24,827 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:24,828 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:24,828 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:24,828 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:24,828 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:24,828 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:24,828 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:24,828 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:24,828 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,828 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. started... 2023-02-13 12:17:24,828 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0497e21f354e69dc75ccda3c9bde659 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:24,833 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,833 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,833 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,833 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:24,833 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,833 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:24,842 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/ad920cca11f4401cbdcf35284e95d8ba 2023-02-13 12:17:24,852 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/15053cb9a39b4b5281c171dda3e07615 2023-02-13 12:17:24,855 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/ad920cca11f4401cbdcf35284e95d8ba as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba 2023-02-13 12:17:24,859 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:17:24,860 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/15053cb9a39b4b5281c171dda3e07615 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615 2023-02-13 12:17:24,863 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 12:17:24,864 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0497e21f354e69dc75ccda3c9bde659 in 36ms, sequenceid=19, compaction requested=false 2023-02-13 12:17:24,864 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0497e21f354e69dc75ccda3c9bde659: 2023-02-13 12:17:24,864 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:24,864 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:24,864 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:24,864 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,864 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:24,864 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:24,875 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,875 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:24,875 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,875 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:24,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 12:17:24,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a6e272a[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:24,877 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 12:17:24,877 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,883 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,883 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,883 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,883 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,883 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:24,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,883 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:24,883 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:24,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,884 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:24,883 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:24,884 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:24,884 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:24,884 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:24,884 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:24,884 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:24,891 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:24,891 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:24,891 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,891 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:24,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:24,891 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:24,892 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:24,891 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:24,892 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:24,891 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:24,892 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:24,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:17:24,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 12:17:24,894 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 12:17:24,894 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:24,894 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:24,894 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 12:17:24,894 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:24,900 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:24,900 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:24,900 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,900 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,901 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,901 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:24,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:24,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,902 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:24,902 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:24,902 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,926 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,926 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,926 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,926 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,926 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 12:17:24,927 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:24,926 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:24,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,926 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,926 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:24,927 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:24,927 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:24,926 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:24,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:24,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:24,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:24,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:24,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,927 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:24,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:24,928 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:24,928 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,027 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:25,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:25,037 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:25,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:25,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 12:17:25,039 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 12:17:25,039 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,039 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 12:17:25,039 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 12:17:25,039 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,039 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,050 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,050 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,050 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,050 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,050 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,050 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,051 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:25,052 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:25,052 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:25,052 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. started... 2023-02-13 12:17:25,052 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0497e21f354e69dc75ccda3c9bde659 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:25,058 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,060 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/578823bbec5147c0b8e199a8470b81f9 2023-02-13 12:17:25,066 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,066 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,066 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,067 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,067 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,071 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/38eb588a2441408b8cf2a9c854c790ef 2023-02-13 12:17:25,075 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/578823bbec5147c0b8e199a8470b81f9 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9 2023-02-13 12:17:25,078 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:17:25,078 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/38eb588a2441408b8cf2a9c854c790ef as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef 2023-02-13 12:17:25,082 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 12:17:25,082 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0497e21f354e69dc75ccda3c9bde659 in 30ms, sequenceid=32, compaction requested=true 2023-02-13 12:17:25,082 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0497e21f354e69dc75ccda3c9bde659: 2023-02-13 12:17:25,082 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:25,082 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:25,082 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:25,082 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,082 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,082 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,091 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,091 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,092 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,092 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 12:17:25,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66e57914[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:25,094 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 12:17:25,094 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,100 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,100 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,100 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,100 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,100 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,100 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,100 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,100 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:25,100 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,100 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:25,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,101 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:25,101 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,101 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,101 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:25,108 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,108 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,108 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,108 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,108 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,108 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,109 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,116 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,117 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,117 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:17:25,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 12:17:25,119 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 12:17:25,119 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:25,119 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:25,119 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 12:17:25,120 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:25,125 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,125 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,125 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,126 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,126 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,127 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,127 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,127 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,133 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,133 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,133 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,133 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,133 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,133 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,133 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 12:17:25,133 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:25,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,133 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,133 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,133 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:25,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,134 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,234 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:25,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:25,244 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:25,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:25,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 12:17:25,246 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 12:17:25,246 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,246 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 12:17:25,246 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 12:17:25,247 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,247 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,257 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,257 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,257 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,257 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,258 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,258 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:25,259 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:25,259 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:25,260 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. started... 2023-02-13 12:17:25,260 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0497e21f354e69dc75ccda3c9bde659 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:25,266 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,266 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,267 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,267 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,268 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,268 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,270 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/697aaa4849ed4137a61dd54de688533e 2023-02-13 12:17:25,285 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc 2023-02-13 12:17:25,288 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/697aaa4849ed4137a61dd54de688533e as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e 2023-02-13 12:17:25,292 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:17:25,293 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc 2023-02-13 12:17:25,296 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 12:17:25,297 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0497e21f354e69dc75ccda3c9bde659 in 37ms, sequenceid=45, compaction requested=true 2023-02-13 12:17:25,297 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0497e21f354e69dc75ccda3c9bde659: 2023-02-13 12:17:25,297 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:25,297 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:25,297 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:25,297 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,297 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,297 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,308 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,308 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,309 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,309 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,310 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 12:17:25,310 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 12:17:25,310 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66f8f3e6[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:25,311 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,316 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,316 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,316 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,317 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,317 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,317 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:25,317 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,317 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,317 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,317 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:25,317 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,317 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,317 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:25,317 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:25,325 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,325 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,325 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,352 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,352 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,352 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,352 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,352 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,353 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,353 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:17:25,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 12:17:25,356 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 12:17:25,356 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:25,356 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:25,356 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 12:17:25,356 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:25,366 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,366 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,366 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,366 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,367 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,367 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,367 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,367 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,367 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,368 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,369 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,369 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,370 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,375 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 12:17:25,376 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:25,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,376 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:25,375 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,376 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,376 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:25,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,376 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,376 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,376 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,376 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,376 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,476 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:25,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:25,487 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:25,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:25,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 12:17:25,488 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 12:17:25,488 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,488 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 12:17:25,488 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 12:17:25,489 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,489 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,500 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,500 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,500 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,500 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,500 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,500 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:25,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,501 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,501 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,501 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,501 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,502 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,502 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:25,502 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:25,502 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:25,502 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. started... 2023-02-13 12:17:25,502 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0497e21f354e69dc75ccda3c9bde659 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:25,508 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,512 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/fd7220829c8841e29c536be118031208 2023-02-13 12:17:25,516 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,516 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,516 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,517 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,517 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,522 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/73c97141e4794a0b9d6db33a7f24d6e5 2023-02-13 12:17:25,525 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/fd7220829c8841e29c536be118031208 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208 2023-02-13 12:17:25,528 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:17:25,529 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/73c97141e4794a0b9d6db33a7f24d6e5 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5 2023-02-13 12:17:25,532 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 12:17:25,533 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0497e21f354e69dc75ccda3c9bde659 in 31ms, sequenceid=58, compaction requested=true 2023-02-13 12:17:25,533 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0497e21f354e69dc75ccda3c9bde659: 2023-02-13 12:17:25,533 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:25,533 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:25,533 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:25,533 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,533 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,533 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,541 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,541 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,542 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,542 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 12:17:25,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@595a1393[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:25,547 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 12:17:25,547 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,557 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,557 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,557 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,557 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,558 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:25,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,558 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,558 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,558 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:25,558 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,558 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,558 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:25,558 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,558 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,558 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:25,566 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,566 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,567 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,575 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,575 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,575 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,575 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,575 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,575 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,575 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:17:25,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 12:17:25,578 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 12:17:25,578 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:25,578 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:25,578 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 12:17:25,578 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:25,583 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,583 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,583 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,583 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,583 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,583 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,583 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,583 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,584 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,584 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,586 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,586 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,586 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,591 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,591 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,591 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,591 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,591 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,591 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,592 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,591 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 12:17:25,592 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:25,591 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,591 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,592 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,592 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,592 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:25,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,592 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,592 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,592 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,593 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,593 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,593 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,593 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,593 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,693 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:25,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:25,704 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:17:25,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 12:17:25,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 12:17:25,705 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 12:17:25,705 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,706 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 12:17:25,706 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 12:17:25,706 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,706 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,715 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,715 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,715 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,715 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,716 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,716 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:17:25,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,716 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,716 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 12:17:25,717 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,717 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,717 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 12:17:25,717 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,717 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,717 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 12:17:25,717 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,717 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,718 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,718 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:17:25,717 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 12:17:25,718 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:17:25,718 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:25,718 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:17:25,718 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,718 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. started... 2023-02-13 12:17:25,718 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 12:17:25,718 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,718 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,718 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33055,1676290533708' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,718 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0497e21f354e69dc75ccda3c9bde659 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 12:17:25,718 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,45345,1676290533799' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,725 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/de01aaac85b242848f8848ded8493532 2023-02-13 12:17:25,736 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0 2023-02-13 12:17:25,739 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/de01aaac85b242848f8848ded8493532 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532 2023-02-13 12:17:25,742 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:17:25,743 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0 2023-02-13 12:17:25,746 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 12:17:25,746 INFO [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0497e21f354e69dc75ccda3c9bde659 in 28ms, sequenceid=71, compaction requested=true 2023-02-13 12:17:25,747 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0497e21f354e69dc75ccda3c9bde659: 2023-02-13 12:17:25,747 DEBUG [rs(jenkins-hbase16.apache.org,33961,1676290533759)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:25,747 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:17:25,747 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:17:25,747 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,747 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 12:17:25,747 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 12:17:25,757 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,775 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,775 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,775 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,783 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,783 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,783 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,783 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,783 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,783 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:17:25,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,33961,1676290533759' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 12:17:25,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@623628e5[Count = 0] remaining members to acquire global barrier 2023-02-13 12:17:25,788 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 12:17:25,788 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,799 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,799 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,799 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,799 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,799 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,799 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:17:25,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,799 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,799 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,799 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,799 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33961,1676290533759' in zk 2023-02-13 12:17:25,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,799 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,45345,1676290533799' in zk 2023-02-13 12:17:25,799 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 12:17:25,800 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 12:17:25,800 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,33055,1676290533708' in zk 2023-02-13 12:17:25,808 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,808 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,808 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,808 DEBUG [member: 'jenkins-hbase16.apache.org,33961,1676290533759' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,816 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,817 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,817 DEBUG [member: 'jenkins-hbase16.apache.org,33055,1676290533708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,816 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 12:17:25,817 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,817 DEBUG [member: 'jenkins-hbase16.apache.org,45345,1676290533799' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 12:17:25,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,33961,1676290533759': 2023-02-13 12:17:25,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,33961,1676290533759' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 12:17:25,819 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 12:17:25,819 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:17:25,819 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:17:25,819 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 12:17:25,819 INFO [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:17:25,825 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,825 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,825 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:17:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:17:25,825 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,825 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,825 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,825 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:17:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,826 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,825 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,826 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:17:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,827 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:17:25,827 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 12:17:25,828 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,833 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,833 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,833 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,833 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,833 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,833 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:17:25,833 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,833 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 12:17:25,833 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:17:25,833 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,833 DEBUG [(jenkins-hbase16.apache.org,39699,1676290532646)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:17:25,833 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:17:25,833 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:17:25,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,834 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 12:17:25,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:17:25,834 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:25,834 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 12:17:25,834 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:17:25,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,834 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:17:25,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:17:25,834 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,834 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:25,834 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 12:17:25,834 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 12:17:25,934 DEBUG [Listener at localhost.localdomain/46087] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:17:25,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:17:25,942 DEBUG [Listener at localhost.localdomain/46087] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,942 DEBUG [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e0497e21f354e69dc75ccda3c9bde659, daughterA=fa58494f1123d750caab1de67257fd8c, daughterB=4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:25,944 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e0497e21f354e69dc75ccda3c9bde659, daughterA=fa58494f1123d750caab1de67257fd8c, daughterB=4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:25,944 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e0497e21f354e69dc75ccda3c9bde659, daughterA=fa58494f1123d750caab1de67257fd8c, daughterB=4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:25,944 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e0497e21f354e69dc75ccda3c9bde659, daughterA=fa58494f1123d750caab1de67257fd8c, daughterB=4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:25,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e0497e21f354e69dc75ccda3c9bde659, UNASSIGN}] 2023-02-13 12:17:25,947 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e0497e21f354e69dc75ccda3c9bde659, UNASSIGN 2023-02-13 12:17:25,947 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=e0497e21f354e69dc75ccda3c9bde659, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:25,947 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290645947"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290645947"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290645947"}]},"ts":"1676290645947"} 2023-02-13 12:17:25,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure e0497e21f354e69dc75ccda3c9bde659, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:17:26,100 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing e0497e21f354e69dc75ccda3c9bde659, disabling compactions & flushes 2023-02-13 12:17:26,100 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:26,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:26,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. after waiting 0 ms 2023-02-13 12:17:26,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:26,101 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing e0497e21f354e69dc75ccda3c9bde659 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 12:17:26,109 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/44f7985be8cb43e490aa8395ca9c224f 2023-02-13 12:17:26,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/f8116acc17294b0598d9e44fde34a4d0 2023-02-13 12:17:26,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf1/44f7985be8cb43e490aa8395ca9c224f as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f 2023-02-13 12:17:26,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:17:26,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/.tmp/cf2/f8116acc17294b0598d9e44fde34a4d0 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0 2023-02-13 12:17:26,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 12:17:26,130 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e0497e21f354e69dc75ccda3c9bde659 in 29ms, sequenceid=82, compaction requested=true 2023-02-13 12:17:26,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 12:17:26,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:26,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. 2023-02-13 12:17:26,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for e0497e21f354e69dc75ccda3c9bde659: Waiting for close lock at 1676290646100Running coprocessor pre-close hooks at 1676290646100Disabling compacts and flushes for region at 1676290646100Disabling writes for close at 1676290646101 (+1 ms)Obtaining lock to block concurrent updates at 1676290646101Preparing flush snapshotting stores in e0497e21f354e69dc75ccda3c9bde659 at 1676290646101Finished memstore snapshotting testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676290646101Flushing stores of testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. at 1676290646101Flushing e0497e21f354e69dc75ccda3c9bde659/cf1: creating writer at 1676290646102 (+1 ms)Flushing e0497e21f354e69dc75ccda3c9bde659/cf1: appending metadata at 1676290646104 (+2 ms)Flushing e0497e21f354e69dc75ccda3c9bde659/cf1: closing flushed file at 1676290646104Flushing e0497e21f354e69dc75ccda3c9bde659/cf2: creating writer at 1676290646112 (+8 ms)Flushing e0497e21f354e69dc75ccda3c9bde659/cf2: appending metadata at 1676290646114 (+2 ms)Flushing e0497e21f354e69dc75ccda3c9bde659/cf2: closing flushed file at 1676290646114Flushing e0497e21f354e69dc75ccda3c9bde659/cf1: reopening flushed file at 1676290646123 (+9 ms)Flushing e0497e21f354e69dc75ccda3c9bde659/cf2: reopening flushed file at 1676290646127 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e0497e21f354e69dc75ccda3c9bde659 in 29ms, sequenceid=82, compaction requested=true at 1676290646130 (+3 ms)Writing region close event to WAL at 1676290646132 (+2 ms)Running coprocessor post-close hooks at 1676290646135 (+3 ms)Closed at 1676290646135 2023-02-13 12:17:26,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,137 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=e0497e21f354e69dc75ccda3c9bde659, regionState=CLOSED 2023-02-13 12:17:26,137 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290646137"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290646137"}]},"ts":"1676290646137"} 2023-02-13 12:17:26,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-13 12:17:26,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure e0497e21f354e69dc75ccda3c9bde659, server=jenkins-hbase16.apache.org,33961,1676290533759 in 190 msec 2023-02-13 12:17:26,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-13 12:17:26,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e0497e21f354e69dc75ccda3c9bde659, UNASSIGN in 193 msec 2023-02-13 12:17:26,149 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=e0497e21f354e69dc75ccda3c9bde659, threads=14 2023-02-13 12:17:26,149 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,149 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,149 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,149 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,149 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,150 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,150 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,150 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,150 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,150 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,150 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,150 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,150 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,150 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,324 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,353 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,356 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,356 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,360 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,366 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,368 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,373 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,381 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,384 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,384 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,384 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,386 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80 for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,390 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f for region: e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:26,390 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region e0497e21f354e69dc75ccda3c9bde659 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 12:17:26,432 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:17:26,433 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 12:17:26,434 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676290646434"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676290646434"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676290646434"}]},"ts":"1676290646434"} 2023-02-13 12:17:26,435 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290646434"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290646434"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290646434"}]},"ts":"1676290646434"} 2023-02-13 12:17:26,435 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290646434"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290646434"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290646434"}]},"ts":"1676290646434"} 2023-02-13 12:17:26,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=fa58494f1123d750caab1de67257fd8c, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=4cc411289d238fc79c19fbad02dac709, ASSIGN}] 2023-02-13 12:17:26,438 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=4cc411289d238fc79c19fbad02dac709, ASSIGN 2023-02-13 12:17:26,438 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=4cc411289d238fc79c19fbad02dac709, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,33961,1676290533759; forceNewPlan=false, retain=false 2023-02-13 12:17:26,439 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=fa58494f1123d750caab1de67257fd8c, ASSIGN 2023-02-13 12:17:26,439 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=fa58494f1123d750caab1de67257fd8c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,33961,1676290533759; forceNewPlan=false, retain=false 2023-02-13 12:17:26,589 INFO [jenkins-hbase16:39699] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 12:17:26,590 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=fa58494f1123d750caab1de67257fd8c, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:26,590 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290646590"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290646590"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290646590"}]},"ts":"1676290646590"} 2023-02-13 12:17:26,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure fa58494f1123d750caab1de67257fd8c, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:17:26,594 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=4cc411289d238fc79c19fbad02dac709, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:26,594 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290646594"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290646594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290646594"}]},"ts":"1676290646594"} 2023-02-13 12:17:26,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 4cc411289d238fc79c19fbad02dac709, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:17:26,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:26,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fa58494f1123d750caab1de67257fd8c, NAME => 'testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 12:17:26,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:17:26,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:26,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:26,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:26,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:26,748 INFO [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:26,749 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1 2023-02-13 12:17:26,749 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1 2023-02-13 12:17:26,749 INFO [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fa58494f1123d750caab1de67257fd8c columnFamilyName cf1 2023-02-13 12:17:26,761 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f-bottom 2023-02-13 12:17:26,764 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9-bottom 2023-02-13 12:17:26,767 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e-bottom 2023-02-13 12:17:26,771 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba-bottom 2023-02-13 12:17:26,774 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532-bottom 2023-02-13 12:17:26,778 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80-bottom 2023-02-13 12:17:26,782 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208-bottom 2023-02-13 12:17:26,782 INFO [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(310): Store=fa58494f1123d750caab1de67257fd8c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:26,782 INFO [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:26,783 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2 2023-02-13 12:17:26,783 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2 2023-02-13 12:17:26,783 INFO [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fa58494f1123d750caab1de67257fd8c columnFamilyName cf2 2023-02-13 12:17:26,794 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615-bottom 2023-02-13 12:17:26,798 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc-bottom 2023-02-13 12:17:26,801 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef-bottom 2023-02-13 12:17:26,805 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5-bottom 2023-02-13 12:17:26,809 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0-bottom 2023-02-13 12:17:26,813 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580-bottom 2023-02-13 12:17:26,816 DEBUG [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0-bottom 2023-02-13 12:17:26,816 INFO [StoreOpener-fa58494f1123d750caab1de67257fd8c-1] regionserver.HStore(310): Store=fa58494f1123d750caab1de67257fd8c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:26,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:26,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:26,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:17:26,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:26,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened fa58494f1123d750caab1de67257fd8c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72006174, jitterRate=0.07297560572624207}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:17:26,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for fa58494f1123d750caab1de67257fd8c: Running coprocessor pre-open hook at 1676290646747Writing region info on filesystem at 1676290646747Initializing all the Stores at 1676290646747Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290646747Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290646748 (+1 ms)Cleaning up temporary data from old regions at 1676290646818 (+70 ms)Cleaning up detritus from prior splits at 1676290646818Running coprocessor post-open hooks at 1676290646820 (+2 ms)Region opened successfully at 1676290646820 2023-02-13 12:17:26,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c., pid=149, masterSystemTime=1676290646743 2023-02-13 12:17:26,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:26,822 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:17:26,822 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:17:26,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:26,822 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:26,822 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:26,822 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1912): fa58494f1123d750caab1de67257fd8c/cf1 is initiating minor compaction (all files) 2023-02-13 12:17:26,822 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1912): fa58494f1123d750caab1de67257fd8c/cf2 is initiating minor compaction (all files) 2023-02-13 12:17:26,822 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of fa58494f1123d750caab1de67257fd8c/cf1 in testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:26,822 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HRegion(2258): Starting compaction of fa58494f1123d750caab1de67257fd8c/cf2 in testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:26,823 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f-bottom] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/.tmp, totalSize=34.9 K 2023-02-13 12:17:26,823 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0-bottom] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/.tmp, totalSize=34.9 K 2023-02-13 12:17:26,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:26,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:26,824 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290644588 2023-02-13 12:17:26,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:26,824 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676290644588 2023-02-13 12:17:26,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4cc411289d238fc79c19fbad02dac709, NAME => 'testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 12:17:26,824 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=fa58494f1123d750caab1de67257fd8c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:26,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 12:17:26,824 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290646824"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290646824"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290646824"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290646824"}]},"ts":"1676290646824"} 2023-02-13 12:17:26,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:26,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:17:26,824 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290644810 2023-02-13 12:17:26,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:26,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:26,825 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676290644810 2023-02-13 12:17:26,825 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290645029 2023-02-13 12:17:26,826 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting 578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676290645029 2023-02-13 12:17:26,826 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290645236 2023-02-13 12:17:26,826 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting 697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676290645236 2023-02-13 12:17:26,826 INFO [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:26,826 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290645478 2023-02-13 12:17:26,826 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676290645478 2023-02-13 12:17:26,827 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290645694 2023-02-13 12:17:26,827 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676290645694 2023-02-13 12:17:26,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-13 12:17:26,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure fa58494f1123d750caab1de67257fd8c, server=jenkins-hbase16.apache.org,33961,1676290533759 in 233 msec 2023-02-13 12:17:26,827 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting 44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290645936 2023-02-13 12:17:26,827 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1 2023-02-13 12:17:26,827 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1 2023-02-13 12:17:26,828 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676290645936 2023-02-13 12:17:26,828 INFO [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4cc411289d238fc79c19fbad02dac709 columnFamilyName cf1 2023-02-13 12:17:26,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=fa58494f1123d750caab1de67257fd8c, ASSIGN in 390 msec 2023-02-13 12:17:26,841 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] throttle.PressureAwareThroughputController(145): fa58494f1123d750caab1de67257fd8c#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:17:26,841 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] throttle.PressureAwareThroughputController(145): fa58494f1123d750caab1de67257fd8c#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:17:26,845 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f-top 2023-02-13 12:17:26,848 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9-top 2023-02-13 12:17:26,851 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e-top 2023-02-13 12:17:26,857 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba-top 2023-02-13 12:17:26,860 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532-top 2023-02-13 12:17:26,863 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80-top 2023-02-13 12:17:26,872 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208-top 2023-02-13 12:17:26,872 INFO [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(310): Store=4cc411289d238fc79c19fbad02dac709/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:26,872 INFO [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:26,873 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2 2023-02-13 12:17:26,873 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2 2023-02-13 12:17:26,873 INFO [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4cc411289d238fc79c19fbad02dac709 columnFamilyName cf2 2023-02-13 12:17:26,875 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/.tmp/cf2/aa2139b125974364ad266a24ba19fd28 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/aa2139b125974364ad266a24ba19fd28 2023-02-13 12:17:26,875 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/.tmp/cf1/bbb64b3db9da428a9db405ac3637bd01 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/bbb64b3db9da428a9db405ac3637bd01 2023-02-13 12:17:26,879 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fa58494f1123d750caab1de67257fd8c/cf1 of fa58494f1123d750caab1de67257fd8c into bbb64b3db9da428a9db405ac3637bd01(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:17:26,879 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fa58494f1123d750caab1de67257fd8c/cf2 of fa58494f1123d750caab1de67257fd8c into aa2139b125974364ad266a24ba19fd28(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:17:26,879 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for fa58494f1123d750caab1de67257fd8c: 2023-02-13 12:17:26,879 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for fa58494f1123d750caab1de67257fd8c: 2023-02-13 12:17:26,879 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c., storeName=fa58494f1123d750caab1de67257fd8c/cf1, priority=9, startTime=1676290646821; duration=0sec 2023-02-13 12:17:26,879 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c., storeName=fa58494f1123d750caab1de67257fd8c/cf2, priority=9, startTime=1676290646822; duration=0sec 2023-02-13 12:17:26,879 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:26,879 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:26,885 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615-top 2023-02-13 12:17:26,888 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc-top 2023-02-13 12:17:26,890 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef-top 2023-02-13 12:17:26,893 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5-top 2023-02-13 12:17:26,896 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0-top 2023-02-13 12:17:26,899 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580-top 2023-02-13 12:17:26,903 DEBUG [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0-top 2023-02-13 12:17:26,903 INFO [StoreOpener-4cc411289d238fc79c19fbad02dac709-1] regionserver.HStore(310): Store=4cc411289d238fc79c19fbad02dac709/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:17:26,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:26,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:26,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:17:26,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:26,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 4cc411289d238fc79c19fbad02dac709; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60512209, jitterRate=-0.09829781949520111}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:17:26,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 4cc411289d238fc79c19fbad02dac709: Running coprocessor pre-open hook at 1676290646825Writing region info on filesystem at 1676290646825Initializing all the Stores at 1676290646826 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290646826Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676290646826Cleaning up temporary data from old regions at 1676290646905 (+79 ms)Cleaning up detritus from prior splits at 1676290646905Running coprocessor post-open hooks at 1676290646907 (+2 ms)Region opened successfully at 1676290646907 2023-02-13 12:17:26,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709., pid=150, masterSystemTime=1676290646743 2023-02-13 12:17:26,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:26,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:17:26,909 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:17:26,909 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 12:17:26,909 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:26,909 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1912): 4cc411289d238fc79c19fbad02dac709/cf1 is initiating minor compaction (all files) 2023-02-13 12:17:26,909 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:26,909 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1912): 4cc411289d238fc79c19fbad02dac709/cf2 is initiating minor compaction (all files) 2023-02-13 12:17:26,909 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4cc411289d238fc79c19fbad02dac709/cf1 in testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:26,909 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4cc411289d238fc79c19fbad02dac709/cf2 in testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:26,910 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0-top] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/.tmp, totalSize=34.9 K 2023-02-13 12:17:26,910 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f-top] into tmpdir=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/.tmp, totalSize=34.9 K 2023-02-13 12:17:26,910 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676290644588 2023-02-13 12:17:26,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:26,910 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676290644588 2023-02-13 12:17:26,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:26,911 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=4cc411289d238fc79c19fbad02dac709, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:26,911 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676290644810 2023-02-13 12:17:26,911 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676290644810 2023-02-13 12:17:26,911 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290646910"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676290646910"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676290646910"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676290646910"}]},"ts":"1676290646910"} 2023-02-13 12:17:26,911 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676290645029 2023-02-13 12:17:26,911 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting 578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676290645029 2023-02-13 12:17:26,911 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676290645236 2023-02-13 12:17:26,911 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting 697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676290645236 2023-02-13 12:17:26,912 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting 73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676290645478 2023-02-13 12:17:26,912 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676290645478 2023-02-13 12:17:26,912 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676290645694 2023-02-13 12:17:26,912 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676290645694 2023-02-13 12:17:26,912 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] compactions.Compactor(207): Compacting f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676290645936 2023-02-13 12:17:26,913 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] compactions.Compactor(207): Compacting 44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676290645936 2023-02-13 12:17:26,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-13 12:17:26,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 4cc411289d238fc79c19fbad02dac709, server=jenkins-hbase16.apache.org,33961,1676290533759 in 317 msec 2023-02-13 12:17:26,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-13 12:17:26,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=4cc411289d238fc79c19fbad02dac709, ASSIGN in 478 msec 2023-02-13 12:17:26,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=e0497e21f354e69dc75ccda3c9bde659, daughterA=fa58494f1123d750caab1de67257fd8c, daughterB=4cc411289d238fc79c19fbad02dac709 in 975 msec 2023-02-13 12:17:26,922 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] throttle.PressureAwareThroughputController(145): 4cc411289d238fc79c19fbad02dac709#cf2#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:17:26,922 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4cc411289d238fc79c19fbad02dac709#cf1#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:17:26,941 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/.tmp/cf2/9aeae90342474a4baf792cd57b8dd22b as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/9aeae90342474a4baf792cd57b8dd22b 2023-02-13 12:17:26,941 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/.tmp/cf1/d0fd048095574e6e89dc96fc32028110 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/d0fd048095574e6e89dc96fc32028110 2023-02-13 12:17:26,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33961] ipc.CallRunner(144): callId: 1120 service: ClientService methodName: Scan size: 143 connection: 188.40.62.62:45990 deadline: 1676290706944, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659. is not online on jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:26,945 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4cc411289d238fc79c19fbad02dac709/cf1 of 4cc411289d238fc79c19fbad02dac709 into d0fd048095574e6e89dc96fc32028110(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:17:26,945 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4cc411289d238fc79c19fbad02dac709/cf2 of 4cc411289d238fc79c19fbad02dac709 into 9aeae90342474a4baf792cd57b8dd22b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:17:26,945 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4cc411289d238fc79c19fbad02dac709: 2023-02-13 12:17:26,945 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4cc411289d238fc79c19fbad02dac709: 2023-02-13 12:17:26,945 INFO [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709., storeName=4cc411289d238fc79c19fbad02dac709/cf1, priority=9, startTime=1676290646908; duration=0sec 2023-02-13 12:17:26,945 INFO [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709., storeName=4cc411289d238fc79c19fbad02dac709/cf2, priority=9, startTime=1676290646909; duration=0sec 2023-02-13 12:17:26,946 DEBUG [RS:1;jenkins-hbase16:33961-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:26,946 DEBUG [RS:1;jenkins-hbase16:33961-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:17:27,054 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 12:17:27,061 INFO [Listener at localhost.localdomain/46087] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 12:17:27,063 DEBUG [Listener at localhost.localdomain/46087] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 12:17:27,072 WARN [Listener at localhost.localdomain/46087] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 12:17:27,074 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-13 12:17:27,074 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitTableRegion 2023-02-13 12:17:27,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-13 12:17:27,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 12:17:27,078 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290647078"}]},"ts":"1676290647078"} 2023-02-13 12:17:27,079 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-13 12:17:27,108 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-13 12:17:27,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=fa58494f1123d750caab1de67257fd8c, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=4cc411289d238fc79c19fbad02dac709, UNASSIGN}] 2023-02-13 12:17:27,110 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=4cc411289d238fc79c19fbad02dac709, UNASSIGN 2023-02-13 12:17:27,110 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=4cc411289d238fc79c19fbad02dac709, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:27,110 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290647110"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290647110"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290647110"}]},"ts":"1676290647110"} 2023-02-13 12:17:27,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 4cc411289d238fc79c19fbad02dac709, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:17:27,113 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=fa58494f1123d750caab1de67257fd8c, UNASSIGN 2023-02-13 12:17:27,113 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=fa58494f1123d750caab1de67257fd8c, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:27,113 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290647113"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676290647113"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676290647113"}]},"ts":"1676290647113"} 2023-02-13 12:17:27,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure fa58494f1123d750caab1de67257fd8c, server=jenkins-hbase16.apache.org,33961,1676290533759}] 2023-02-13 12:17:27,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 12:17:27,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:27,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing fa58494f1123d750caab1de67257fd8c, disabling compactions & flushes 2023-02-13 12:17:27,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:27,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:27,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. after waiting 0 ms 2023-02-13 12:17:27,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:27,264 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f-bottom] to archive 2023-02-13 12:17:27,264 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:17:27,266 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,267 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,268 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,269 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,270 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,271 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,272 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,273 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0-bottom, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0-bottom] to archive 2023-02-13 12:17:27,273 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:17:27,274 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,275 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,276 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,277 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,278 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,279 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,280 DEBUG [StoreCloser-testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 12:17:27,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:27,283 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c. 2023-02-13 12:17:27,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for fa58494f1123d750caab1de67257fd8c: Waiting for close lock at 1676290647263Running coprocessor pre-close hooks at 1676290647263Disabling compacts and flushes for region at 1676290647263Disabling writes for close at 1676290647263Writing region close event to WAL at 1676290647281 (+18 ms)Running coprocessor post-close hooks at 1676290647283 (+2 ms)Closed at 1676290647283 2023-02-13 12:17:27,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:27,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:27,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 4cc411289d238fc79c19fbad02dac709, disabling compactions & flushes 2023-02-13 12:17:27,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:27,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:27,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. after waiting 0 ms 2023-02-13 12:17:27,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:27,285 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=fa58494f1123d750caab1de67257fd8c, regionState=CLOSED 2023-02-13 12:17:27,285 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290647285"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290647285"}]},"ts":"1676290647285"} 2023-02-13 12:17:27,286 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f-top] to archive 2023-02-13 12:17:27,286 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:17:27,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-13 12:17:27,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure fa58494f1123d750caab1de67257fd8c, server=jenkins-hbase16.apache.org,33961,1676290533759 in 172 msec 2023-02-13 12:17:27,288 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/e8f15f3829c64a0caad4220fa3303a80.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=fa58494f1123d750caab1de67257fd8c, UNASSIGN in 179 msec 2023-02-13 12:17:27,289 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/ad920cca11f4401cbdcf35284e95d8ba.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,290 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/578823bbec5147c0b8e199a8470b81f9.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,291 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/697aaa4849ed4137a61dd54de688533e.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,291 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/fd7220829c8841e29c536be118031208.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,292 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/de01aaac85b242848f8848ded8493532.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,293 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/44f7985be8cb43e490aa8395ca9c224f.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,295 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0-top, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659->hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0-top] to archive 2023-02-13 12:17:27,296 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:17:27,297 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/f8c192c06cac430ba4b13bf714ad2580.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,298 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/15053cb9a39b4b5281c171dda3e07615.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,299 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/38eb588a2441408b8cf2a9c854c790ef.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,301 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,302 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/73c97141e4794a0b9d6db33a7f24d6e5.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,303 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,304 DEBUG [StoreCloser-testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/f8116acc17294b0598d9e44fde34a4d0.e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,307 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 12:17:27,308 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:27,308 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709. 2023-02-13 12:17:27,308 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 4cc411289d238fc79c19fbad02dac709: Waiting for close lock at 1676290647285Running coprocessor pre-close hooks at 1676290647285Disabling compacts and flushes for region at 1676290647285Disabling writes for close at 1676290647285Writing region close event to WAL at 1676290647305 (+20 ms)Running coprocessor post-close hooks at 1676290647308 (+3 ms)Closed at 1676290647308 2023-02-13 12:17:27,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:27,309 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=4cc411289d238fc79c19fbad02dac709, regionState=CLOSED 2023-02-13 12:17:27,310 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676290647309"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676290647309"}]},"ts":"1676290647309"} 2023-02-13 12:17:27,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-13 12:17:27,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 4cc411289d238fc79c19fbad02dac709, server=jenkins-hbase16.apache.org,33961,1676290533759 in 199 msec 2023-02-13 12:17:27,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-02-13 12:17:27,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=4cc411289d238fc79c19fbad02dac709, UNASSIGN in 203 msec 2023-02-13 12:17:27,313 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676290647313"}]},"ts":"1676290647313"} 2023-02-13 12:17:27,314 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-13 12:17:27,325 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-13 12:17:27,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 251 msec 2023-02-13 12:17:27,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 12:17:27,381 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-13 12:17:27,382 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitTableRegion 2023-02-13 12:17:27,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 12:17:27,384 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 12:17:27,384 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 12:17:27,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 12:17:27,388 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,389 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:27,388 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:27,390 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/recovered.edits] 2023-02-13 12:17:27,390 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/recovered.edits] 2023-02-13 12:17:27,390 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2, FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/recovered.edits] 2023-02-13 12:17:27,393 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/bbb64b3db9da428a9db405ac3637bd01 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf1/bbb64b3db9da428a9db405ac3637bd01 2023-02-13 12:17:27,393 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/d0fd048095574e6e89dc96fc32028110 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf1/d0fd048095574e6e89dc96fc32028110 2023-02-13 12:17:27,393 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/44f7985be8cb43e490aa8395ca9c224f 2023-02-13 12:17:27,395 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/578823bbec5147c0b8e199a8470b81f9 2023-02-13 12:17:27,395 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/9aeae90342474a4baf792cd57b8dd22b to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/cf2/9aeae90342474a4baf792cd57b8dd22b 2023-02-13 12:17:27,395 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/aa2139b125974364ad266a24ba19fd28 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/cf2/aa2139b125974364ad266a24ba19fd28 2023-02-13 12:17:27,396 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/697aaa4849ed4137a61dd54de688533e 2023-02-13 12:17:27,397 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/recovered.edits/90.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709/recovered.edits/90.seqid 2023-02-13 12:17:27,397 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/ad920cca11f4401cbdcf35284e95d8ba 2023-02-13 12:17:27,397 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/recovered.edits/90.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c/recovered.edits/90.seqid 2023-02-13 12:17:27,397 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/4cc411289d238fc79c19fbad02dac709 2023-02-13 12:17:27,397 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/fa58494f1123d750caab1de67257fd8c 2023-02-13 12:17:27,398 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/de01aaac85b242848f8848ded8493532 2023-02-13 12:17:27,399 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/e8f15f3829c64a0caad4220fa3303a80 2023-02-13 12:17:27,400 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf1/fd7220829c8841e29c536be118031208 2023-02-13 12:17:27,402 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/15053cb9a39b4b5281c171dda3e07615 2023-02-13 12:17:27,403 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/2c3e3f4fee1b4ad4a8aff06455d028dc 2023-02-13 12:17:27,404 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/38eb588a2441408b8cf2a9c854c790ef 2023-02-13 12:17:27,405 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/73c97141e4794a0b9d6db33a7f24d6e5 2023-02-13 12:17:27,406 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8116acc17294b0598d9e44fde34a4d0 2023-02-13 12:17:27,407 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/f8c192c06cac430ba4b13bf714ad2580 2023-02-13 12:17:27,408 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0 to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/cf2/ffdaa9aac8e54b0eb5ea03e1376a5de0 2023-02-13 12:17:27,410 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/recovered.edits/85.seqid to hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/archive/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659/recovered.edits/85.seqid 2023-02-13 12:17:27,410 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/.tmp/data/default/testSplitTableRegion/e0497e21f354e69dc75ccda3c9bde659 2023-02-13 12:17:27,410 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 12:17:27,412 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 12:17:27,413 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-13 12:17:27,415 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-13 12:17:27,416 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 12:17:27,416 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-13 12:17:27,416 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290647416"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:27,416 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290647416"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:27,416 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676290647416"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:27,418 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 12:17:27,418 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e0497e21f354e69dc75ccda3c9bde659, NAME => 'testSplitTableRegion,,1676290644084.e0497e21f354e69dc75ccda3c9bde659.', STARTKEY => '', ENDKEY => ''}, {ENCODED => fa58494f1123d750caab1de67257fd8c, NAME => 'testSplitTableRegion,,1676290645942.fa58494f1123d750caab1de67257fd8c.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 4cc411289d238fc79c19fbad02dac709, NAME => 'testSplitTableRegion,41,1676290645942.4cc411289d238fc79c19fbad02dac709.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 12:17:27,418 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-13 12:17:27,418 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676290647418"}]},"ts":"9223372036854775807"} 2023-02-13 12:17:27,419 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-13 12:17:27,425 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 12:17:27,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 43 msec 2023-02-13 12:17:27,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39699] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 12:17:27,486 INFO [Listener at localhost.localdomain/46087] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-13 12:17:27,499 INFO [Listener at localhost.localdomain/46087] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=379 (was 376) - Thread LEAK? -, OpenFileDescriptor=841 (was 839) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=275 (was 247) - SystemLoadAverage LEAK? -, ProcessCount=178 (was 167) - ProcessCount LEAK? -, AvailableMemoryMB=5332 (was 5358) 2023-02-13 12:17:27,499 INFO [Listener at localhost.localdomain/46087] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 12:17:27,500 INFO [Listener at localhost.localdomain/46087] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:17:27,500 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x42379693 to 127.0.0.1:54988 2023-02-13 12:17:27,500 DEBUG [Listener at localhost.localdomain/46087] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:17:27,662 DEBUG [Listener at localhost.localdomain/46087] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 12:17:27,662 DEBUG [Listener at localhost.localdomain/46087] util.JVMClusterUtil(257): Found active master hash=1696685032, stopped=false 2023-02-13 12:17:27,662 INFO [Listener at localhost.localdomain/46087] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase16.apache.org,39699,1676290532646 2023-02-13 12:17:27,666 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 12:17:27,666 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 12:17:27,666 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 12:17:27,666 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 12:17:27,667 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:17:27,666 INFO [Listener at localhost.localdomain/46087] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:17:27,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:17:27,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:17:27,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:17:27,667 DEBUG [Listener at localhost.localdomain/46087] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x55d858b9 to 127.0.0.1:54988 2023-02-13 12:17:27,667 DEBUG [Listener at localhost.localdomain/46087] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:17:27,667 INFO [Listener at localhost.localdomain/46087] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase16.apache.org,33055,1676290533708' ***** 2023-02-13 12:17:27,667 INFO [Listener at localhost.localdomain/46087] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 12:17:27,668 INFO [Listener at localhost.localdomain/46087] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase16.apache.org,33961,1676290533759' ***** 2023-02-13 12:17:27,668 INFO [Listener at localhost.localdomain/46087] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 12:17:27,668 INFO [Listener at localhost.localdomain/46087] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase16.apache.org,45345,1676290533799' ***** 2023-02-13 12:17:27,668 INFO [Listener at localhost.localdomain/46087] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 12:17:27,668 INFO [RS:0;jenkins-hbase16:33055] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 12:17:27,668 INFO [RS:2;jenkins-hbase16:45345] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 12:17:27,668 INFO [RS:0;jenkins-hbase16:33055] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 12:17:27,668 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 12:17:27,668 INFO [RS:0;jenkins-hbase16:33055] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 12:17:27,668 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 12:17:27,668 INFO [RS:2;jenkins-hbase16:45345] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 12:17:27,668 INFO [RS:2;jenkins-hbase16:45345] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 12:17:27,668 INFO [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(3304): Received CLOSE for 97396b7f803d52a54325415167dadd83 2023-02-13 12:17:27,668 INFO [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(1145): stopping server jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:27,669 DEBUG [RS:2;jenkins-hbase16:45345] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3ce9878e to 127.0.0.1:54988 2023-02-13 12:17:27,669 DEBUG [RS:2;jenkins-hbase16:45345] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:17:27,669 INFO [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(1171): stopping server jenkins-hbase16.apache.org,45345,1676290533799; all regions closed. 2023-02-13 12:17:27,669 INFO [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1145): stopping server jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:27,669 DEBUG [RS:0;jenkins-hbase16:33055] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4a8de98e to 127.0.0.1:54988 2023-02-13 12:17:27,669 INFO [RS:1;jenkins-hbase16:33961] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 12:17:27,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 97396b7f803d52a54325415167dadd83, disabling compactions & flushes 2023-02-13 12:17:27,669 INFO [RS:1;jenkins-hbase16:33961] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 12:17:27,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:17:27,669 DEBUG [RS:0;jenkins-hbase16:33055] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:17:27,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:17:27,669 INFO [RS:1;jenkins-hbase16:33961] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 12:17:27,669 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 12:17:27,669 INFO [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1145): stopping server jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:27,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. after waiting 0 ms 2023-02-13 12:17:27,669 DEBUG [RS:1;jenkins-hbase16:33961] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3155f0d9 to 127.0.0.1:54988 2023-02-13 12:17:27,669 INFO [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 12:17:27,669 DEBUG [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1479): Online Regions={97396b7f803d52a54325415167dadd83=hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83.} 2023-02-13 12:17:27,669 DEBUG [RS:1;jenkins-hbase16:33961] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:17:27,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:17:27,670 INFO [RS:1;jenkins-hbase16:33961] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 12:17:27,670 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 97396b7f803d52a54325415167dadd83 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 12:17:27,670 INFO [RS:1;jenkins-hbase16:33961] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 12:17:27,670 INFO [RS:1;jenkins-hbase16:33961] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 12:17:27,670 INFO [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 12:17:27,670 DEBUG [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1505): Waiting on 97396b7f803d52a54325415167dadd83 2023-02-13 12:17:27,670 INFO [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 12:17:27,670 DEBUG [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-13 12:17:27,670 DEBUG [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 12:17:27,670 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 12:17:27,671 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 12:17:27,671 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 12:17:27,671 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 12:17:27,671 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 12:17:27,671 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-02-13 12:17:27,673 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:17:27,682 DEBUG [RS:2;jenkins-hbase16:45345] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/oldWALs 2023-02-13 12:17:27,682 INFO [RS:2;jenkins-hbase16:45345] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C45345%2C1676290533799:(num 1676290535345) 2023-02-13 12:17:27,682 DEBUG [RS:2;jenkins-hbase16:45345] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:17:27,682 INFO [RS:2;jenkins-hbase16:45345] regionserver.LeaseManager(133): Closed leases 2023-02-13 12:17:27,684 INFO [RS:2;jenkins-hbase16:45345] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 12:17:27,684 INFO [RS:2;jenkins-hbase16:45345] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 12:17:27,684 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 12:17:27,684 INFO [RS:2;jenkins-hbase16:45345] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 12:17:27,684 INFO [RS:2;jenkins-hbase16:45345] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 12:17:27,692 INFO [RS:2;jenkins-hbase16:45345] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:45345 2023-02-13 12:17:27,700 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/namespace/97396b7f803d52a54325415167dadd83/.tmp/info/874aada8867545ea892275f5006b9b04 2023-02-13 12:17:27,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/namespace/97396b7f803d52a54325415167dadd83/.tmp/info/874aada8867545ea892275f5006b9b04 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/namespace/97396b7f803d52a54325415167dadd83/info/874aada8867545ea892275f5006b9b04 2023-02-13 12:17:27,710 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/namespace/97396b7f803d52a54325415167dadd83/info/874aada8867545ea892275f5006b9b04, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 12:17:27,711 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 97396b7f803d52a54325415167dadd83 in 41ms, sequenceid=6, compaction requested=false 2023-02-13 12:17:27,711 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/.tmp/info/e495c559f57746f7a96e2543e14b0d52 2023-02-13 12:17:27,715 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e495c559f57746f7a96e2543e14b0d52 2023-02-13 12:17:27,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/namespace/97396b7f803d52a54325415167dadd83/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 12:17:27,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:27,720 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:17:27,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 97396b7f803d52a54325415167dadd83: Waiting for close lock at 1676290647669Running coprocessor pre-close hooks at 1676290647669Disabling compacts and flushes for region at 1676290647669Disabling writes for close at 1676290647669Obtaining lock to block concurrent updates at 1676290647670 (+1 ms)Preparing flush snapshotting stores in 97396b7f803d52a54325415167dadd83 at 1676290647670Finished memstore snapshotting hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676290647670Flushing stores of hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. at 1676290647671 (+1 ms)Flushing 97396b7f803d52a54325415167dadd83/info: creating writer at 1676290647671Flushing 97396b7f803d52a54325415167dadd83/info: appending metadata at 1676290647678 (+7 ms)Flushing 97396b7f803d52a54325415167dadd83/info: closing flushed file at 1676290647678Flushing 97396b7f803d52a54325415167dadd83/info: reopening flushed file at 1676290647705 (+27 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 97396b7f803d52a54325415167dadd83 in 41ms, sequenceid=6, compaction requested=false at 1676290647711 (+6 ms)Writing region close event to WAL at 1676290647716 (+5 ms)Running coprocessor post-close hooks at 1676290647719 (+3 ms)Closed at 1676290647720 (+1 ms) 2023-02-13 12:17:27,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676290535745.97396b7f803d52a54325415167dadd83. 2023-02-13 12:17:27,729 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/.tmp/rep_barrier/204c34c9382742bf89b43578dd0a5086 2023-02-13 12:17:27,732 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 204c34c9382742bf89b43578dd0a5086 2023-02-13 12:17:27,733 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:27,733 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 12:17:27,733 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:27,733 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 12:17:27,733 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 12:17:27,733 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,45345,1676290533799 2023-02-13 12:17:27,733 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 12:17:27,733 ERROR [Listener at localhost.localdomain/46087-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@735e9c43 rejected from java.util.concurrent.ThreadPoolExecutor@776116f0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:17:27,742 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,45345,1676290533799] 2023-02-13 12:17:27,742 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,45345,1676290533799; numProcessing=1 2023-02-13 12:17:27,746 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/.tmp/table/9140c9394afa433e8640b503bab9c8ae 2023-02-13 12:17:27,750 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,45345,1676290533799 already deleted, retry=false 2023-02-13 12:17:27,750 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9140c9394afa433e8640b503bab9c8ae 2023-02-13 12:17:27,750 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,45345,1676290533799 expired; onlineServers=2 2023-02-13 12:17:27,750 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/.tmp/info/e495c559f57746f7a96e2543e14b0d52 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/info/e495c559f57746f7a96e2543e14b0d52 2023-02-13 12:17:27,754 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e495c559f57746f7a96e2543e14b0d52 2023-02-13 12:17:27,754 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/info/e495c559f57746f7a96e2543e14b0d52, entries=59, sequenceid=210, filesize=11.1 K 2023-02-13 12:17:27,755 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/.tmp/rep_barrier/204c34c9382742bf89b43578dd0a5086 as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/rep_barrier/204c34c9382742bf89b43578dd0a5086 2023-02-13 12:17:27,758 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 204c34c9382742bf89b43578dd0a5086 2023-02-13 12:17:27,759 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/rep_barrier/204c34c9382742bf89b43578dd0a5086, entries=24, sequenceid=210, filesize=7.5 K 2023-02-13 12:17:27,759 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/.tmp/table/9140c9394afa433e8640b503bab9c8ae as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/table/9140c9394afa433e8640b503bab9c8ae 2023-02-13 12:17:27,761 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 12:17:27,761 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 12:17:27,761 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 12:17:27,763 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9140c9394afa433e8640b503bab9c8ae 2023-02-13 12:17:27,763 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/table/9140c9394afa433e8640b503bab9c8ae, entries=36, sequenceid=210, filesize=8.0 K 2023-02-13 12:17:27,765 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 93ms, sequenceid=210, compaction requested=false 2023-02-13 12:17:27,769 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-13 12:17:27,769 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 12:17:27,769 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 12:17:27,770 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 12:17:27,770 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676290647670Running coprocessor pre-close hooks at 1676290647670Disabling compacts and flushes for region at 1676290647670Disabling writes for close at 1676290647671 (+1 ms)Obtaining lock to block concurrent updates at 1676290647671Preparing flush snapshotting stores in 1588230740 at 1676290647671Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81396, getHeapSize=128792, getOffHeapSize=0, getCellsCount=617 at 1676290647671Flushing stores of hbase:meta,,1.1588230740 at 1676290647672 (+1 ms)Flushing 1588230740/info: creating writer at 1676290647673 (+1 ms)Flushing 1588230740/info: appending metadata at 1676290647680 (+7 ms)Flushing 1588230740/info: closing flushed file at 1676290647680Flushing 1588230740/rep_barrier: creating writer at 1676290647716 (+36 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676290647718 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676290647718Flushing 1588230740/table: creating writer at 1676290647732 (+14 ms)Flushing 1588230740/table: appending metadata at 1676290647735 (+3 ms)Flushing 1588230740/table: closing flushed file at 1676290647735Flushing 1588230740/info: reopening flushed file at 1676290647751 (+16 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676290647756 (+5 ms)Flushing 1588230740/table: reopening flushed file at 1676290647760 (+4 ms)Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 93ms, sequenceid=210, compaction requested=false at 1676290647765 (+5 ms)Writing region close event to WAL at 1676290647766 (+1 ms)Running coprocessor post-close hooks at 1676290647769 (+3 ms)Closed at 1676290647770 (+1 ms) 2023-02-13 12:17:27,770 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 12:17:27,842 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:17:27,842 INFO [RS:2;jenkins-hbase16:45345] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase16.apache.org,45345,1676290533799; zookeeper connection closed. 2023-02-13 12:17:27,842 ERROR [Listener at localhost.localdomain/46087-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@42098c9 rejected from java.util.concurrent.ThreadPoolExecutor@776116f0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:17:27,842 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:45345-0x10052d110940003, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:17:27,842 ERROR [Listener at localhost.localdomain/46087-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7cb2f08f rejected from java.util.concurrent.ThreadPoolExecutor@776116f0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:17:27,842 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@66a04cb1] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@66a04cb1 2023-02-13 12:17:27,870 INFO [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1171): stopping server jenkins-hbase16.apache.org,33055,1676290533708; all regions closed. 2023-02-13 12:17:27,870 INFO [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1171): stopping server jenkins-hbase16.apache.org,33961,1676290533759; all regions closed. 2023-02-13 12:17:27,878 DEBUG [RS:0;jenkins-hbase16:33055] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/oldWALs 2023-02-13 12:17:27,879 INFO [RS:0;jenkins-hbase16:33055] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C33055%2C1676290533708:(num 1676290535344) 2023-02-13 12:17:27,879 DEBUG [RS:0;jenkins-hbase16:33055] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:17:27,879 INFO [RS:0;jenkins-hbase16:33055] regionserver.LeaseManager(133): Closed leases 2023-02-13 12:17:27,879 INFO [RS:0;jenkins-hbase16:33055] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 12:17:27,880 INFO [RS:0;jenkins-hbase16:33055] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 12:17:27,881 INFO [RS:0;jenkins-hbase16:33055] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 12:17:27,881 INFO [RS:0;jenkins-hbase16:33055] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 12:17:27,880 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 12:17:27,882 INFO [RS:0;jenkins-hbase16:33055] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:33055 2023-02-13 12:17:27,884 DEBUG [RS:1;jenkins-hbase16:33961] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/oldWALs 2023-02-13 12:17:27,884 INFO [RS:1;jenkins-hbase16:33961] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C33961%2C1676290533759.meta:.meta(num 1676290535462) 2023-02-13 12:17:27,891 DEBUG [RS:1;jenkins-hbase16:33961] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/oldWALs 2023-02-13 12:17:27,891 INFO [RS:1;jenkins-hbase16:33961] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C33961%2C1676290533759:(num 1676290535344) 2023-02-13 12:17:27,891 DEBUG [RS:1;jenkins-hbase16:33961] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:17:27,891 INFO [RS:1;jenkins-hbase16:33961] regionserver.LeaseManager(133): Closed leases 2023-02-13 12:17:27,891 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 12:17:27,891 INFO [RS:1;jenkins-hbase16:33961] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 12:17:27,892 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:27,892 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 12:17:27,891 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,33055,1676290533708 2023-02-13 12:17:27,893 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,33055,1676290533708] 2023-02-13 12:17:27,893 INFO [RS:1;jenkins-hbase16:33961] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:33961 2023-02-13 12:17:27,893 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,33055,1676290533708; numProcessing=2 2023-02-13 12:17:27,908 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,33961,1676290533759 2023-02-13 12:17:27,908 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,33055,1676290533708 already deleted, retry=false 2023-02-13 12:17:27,908 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 12:17:27,908 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,33055,1676290533708 expired; onlineServers=1 2023-02-13 12:17:27,908 ERROR [Listener at localhost.localdomain/46087-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3a755694 rejected from java.util.concurrent.ThreadPoolExecutor@3e4ad502[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:17:27,916 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,33961,1676290533759] 2023-02-13 12:17:27,916 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,33961,1676290533759; numProcessing=3 2023-02-13 12:17:27,925 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,33961,1676290533759 already deleted, retry=false 2023-02-13 12:17:27,925 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,33961,1676290533759 expired; onlineServers=0 2023-02-13 12:17:27,925 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase16.apache.org,39699,1676290532646' ***** 2023-02-13 12:17:27,925 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 12:17:27,926 DEBUG [M:0;jenkins-hbase16:39699] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@241c51d3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-02-13 12:17:27,926 INFO [M:0;jenkins-hbase16:39699] regionserver.HRegionServer(1145): stopping server jenkins-hbase16.apache.org,39699,1676290532646 2023-02-13 12:17:27,926 INFO [M:0;jenkins-hbase16:39699] regionserver.HRegionServer(1171): stopping server jenkins-hbase16.apache.org,39699,1676290532646; all regions closed. 2023-02-13 12:17:27,926 DEBUG [M:0;jenkins-hbase16:39699] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:17:27,926 DEBUG [M:0;jenkins-hbase16:39699] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 12:17:27,927 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 12:17:27,927 DEBUG [M:0;jenkins-hbase16:39699] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 12:17:27,927 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1676290534931] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1676290534931,5,FailOnTimeoutGroup] 2023-02-13 12:17:27,927 INFO [M:0;jenkins-hbase16:39699] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 12:17:27,927 INFO [M:0;jenkins-hbase16:39699] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 12:17:27,927 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1676290534932] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1676290534932,5,FailOnTimeoutGroup] 2023-02-13 12:17:27,927 INFO [M:0;jenkins-hbase16:39699] hbase.ChoreService(369): Chore service for: master/jenkins-hbase16:0 had [] on shutdown 2023-02-13 12:17:27,927 DEBUG [M:0;jenkins-hbase16:39699] master.HMaster(1502): Stopping service threads 2023-02-13 12:17:27,927 INFO [M:0;jenkins-hbase16:39699] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 12:17:27,928 ERROR [M:0;jenkins-hbase16:39699] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-13 12:17:27,928 INFO [M:0;jenkins-hbase16:39699] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:17:27,928 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 12:17:27,933 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 12:17:27,933 DEBUG [M:0;jenkins-hbase16:39699] zookeeper.ZKUtil(398): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 12:17:27,933 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:17:27,933 WARN [M:0;jenkins-hbase16:39699] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 12:17:27,933 INFO [M:0;jenkins-hbase16:39699] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:17:27,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 12:17:27,933 INFO [M:0;jenkins-hbase16:39699] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 12:17:27,934 DEBUG [M:0;jenkins-hbase16:39699] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 12:17:27,934 INFO [M:0;jenkins-hbase16:39699] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:17:27,934 DEBUG [M:0;jenkins-hbase16:39699] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:17:27,934 DEBUG [M:0;jenkins-hbase16:39699] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 12:17:27,934 DEBUG [M:0;jenkins-hbase16:39699] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:17:27,934 INFO [M:0;jenkins-hbase16:39699] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.79 KB heapSize=719.67 KB 2023-02-13 12:17:27,949 INFO [M:0;jenkins-hbase16:39699] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.79 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/fd1d15d420c34ca49e789b3f6a5884bf 2023-02-13 12:17:27,952 INFO [M:0;jenkins-hbase16:39699] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd1d15d420c34ca49e789b3f6a5884bf 2023-02-13 12:17:27,953 DEBUG [M:0;jenkins-hbase16:39699] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/fd1d15d420c34ca49e789b3f6a5884bf as hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/fd1d15d420c34ca49e789b3f6a5884bf 2023-02-13 12:17:27,957 INFO [M:0;jenkins-hbase16:39699] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd1d15d420c34ca49e789b3f6a5884bf 2023-02-13 12:17:27,957 INFO [M:0;jenkins-hbase16:39699] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44273/user/jenkins/test-data/f02fa0a9-8ab0-2eee-f2c8-7d06080c6aa8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/fd1d15d420c34ca49e789b3f6a5884bf, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-13 12:17:27,958 INFO [M:0;jenkins-hbase16:39699] regionserver.HRegion(2947): Finished flush of dataSize ~608.79 KB/623404, heapSize ~719.66 KB/736928, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=1282, compaction requested=false 2023-02-13 12:17:27,959 INFO [M:0;jenkins-hbase16:39699] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:17:27,959 DEBUG [M:0;jenkins-hbase16:39699] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676290647934Disabling compacts and flushes for region at 1676290647934Disabling writes for close at 1676290647934Obtaining lock to block concurrent updates at 1676290647934Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676290647934Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=623404, getHeapSize=736928, getOffHeapSize=0, getCellsCount=1499 at 1676290647934Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676290647935 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676290647935Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676290647939 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676290647939Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676290647954 (+15 ms)Finished flush of dataSize ~608.79 KB/623404, heapSize ~719.66 KB/736928, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=1282, compaction requested=false at 1676290647958 (+4 ms)Writing region close event to WAL at 1676290647959 (+1 ms)Closed at 1676290647959 2023-02-13 12:17:27,965 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 12:17:27,965 INFO [M:0;jenkins-hbase16:39699] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 12:17:27,965 INFO [M:0;jenkins-hbase16:39699] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:39699 2023-02-13 12:17:27,975 DEBUG [M:0;jenkins-hbase16:39699] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase16.apache.org,39699,1676290532646 already deleted, retry=false 2023-02-13 12:17:28,000 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:17:28,000 INFO [RS:0;jenkins-hbase16:33055] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase16.apache.org,33055,1676290533708; zookeeper connection closed. 2023-02-13 12:17:28,000 ERROR [Listener at localhost.localdomain/46087-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@22eda08c rejected from java.util.concurrent.ThreadPoolExecutor@26d22009[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:17:28,000 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5cf13746] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5cf13746 2023-02-13 12:17:28,000 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33055-0x10052d110940001, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:17:28,000 ERROR [Listener at localhost.localdomain/46087-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1d5f5437 rejected from java.util.concurrent.ThreadPoolExecutor@26d22009[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:17:28,100 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:17:28,100 INFO [M:0;jenkins-hbase16:39699] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase16.apache.org,39699,1676290532646; zookeeper connection closed. 2023-02-13 12:17:28,100 ERROR [Listener at localhost.localdomain/46087-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@f4c5511 rejected from java.util.concurrent.ThreadPoolExecutor@21347f1d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:17:28,101 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): master:39699-0x10052d110940000, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:17:28,101 ERROR [Listener at localhost.localdomain/46087-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@55f77b7c rejected from java.util.concurrent.ThreadPoolExecutor@21347f1d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:17:28,201 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:17:28,201 INFO [RS:1;jenkins-hbase16:33961] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase16.apache.org,33961,1676290533759; zookeeper connection closed. 2023-02-13 12:17:28,201 ERROR [Listener at localhost.localdomain/46087-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5a283c9a rejected from java.util.concurrent.ThreadPoolExecutor@3e4ad502[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:17:28,201 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@d810e0d] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@d810e0d 2023-02-13 12:17:28,201 DEBUG [Listener at localhost.localdomain/46087-EventThread] zookeeper.ZKWatcher(600): regionserver:33961-0x10052d110940002, quorum=127.0.0.1:54988, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:17:28,201 ERROR [Listener at localhost.localdomain/46087-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@22f33e35 rejected from java.util.concurrent.ThreadPoolExecutor@3e4ad502[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:17:28,201 INFO [Listener at localhost.localdomain/46087] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-13 12:17:28,202 WARN [Listener at localhost.localdomain/46087] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 12:17:28,235 INFO [Listener at localhost.localdomain/46087] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 12:17:28,342 WARN [BP-591480327-188.40.62.62-1676290528441 heartbeating to localhost.localdomain/127.0.0.1:44273] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 12:17:28,342 WARN [BP-591480327-188.40.62.62-1676290528441 heartbeating to localhost.localdomain/127.0.0.1:44273] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-591480327-188.40.62.62-1676290528441 (Datanode Uuid c0f6d6b4-56b4-4c38-a07d-ddec5113cb94) service to localhost.localdomain/127.0.0.1:44273 2023-02-13 12:17:28,345 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/dfs/data/data5/current/BP-591480327-188.40.62.62-1676290528441] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 12:17:28,345 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/dfs/data/data6/current/BP-591480327-188.40.62.62-1676290528441] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 12:17:28,347 WARN [Listener at localhost.localdomain/46087] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 12:17:28,350 INFO [Listener at localhost.localdomain/46087] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 12:17:28,454 WARN [BP-591480327-188.40.62.62-1676290528441 heartbeating to localhost.localdomain/127.0.0.1:44273] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 12:17:28,455 WARN [BP-591480327-188.40.62.62-1676290528441 heartbeating to localhost.localdomain/127.0.0.1:44273] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-591480327-188.40.62.62-1676290528441 (Datanode Uuid 32c2d397-91bb-4523-b299-5dcd9d7d748d) service to localhost.localdomain/127.0.0.1:44273 2023-02-13 12:17:28,456 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/dfs/data/data3/current/BP-591480327-188.40.62.62-1676290528441] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 12:17:28,456 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/dfs/data/data4/current/BP-591480327-188.40.62.62-1676290528441] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 12:17:28,458 WARN [Listener at localhost.localdomain/46087] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 12:17:28,461 INFO [Listener at localhost.localdomain/46087] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 12:17:28,472 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:17:28,563 WARN [BP-591480327-188.40.62.62-1676290528441 heartbeating to localhost.localdomain/127.0.0.1:44273] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-591480327-188.40.62.62-1676290528441 (Datanode Uuid 14b0d84a-bcac-40f1-8e55-f251954dc488) service to localhost.localdomain/127.0.0.1:44273 2023-02-13 12:17:28,564 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/dfs/data/data1/current/BP-591480327-188.40.62.62-1676290528441] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 12:17:28,564 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0625287f-2c6c-a9e8-7909-c4e46fa6fbfa/cluster_ae5e35cc-f821-0834-a34e-896343c454fa/dfs/data/data2/current/BP-591480327-188.40.62.62-1676290528441] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 12:17:28,584 INFO [Listener at localhost.localdomain/46087] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-02-13 12:17:28,763 INFO [Listener at localhost.localdomain/46087] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 12:17:28,989 INFO [Listener at localhost.localdomain/46087] hbase.HBaseTestingUtility(1293): Minicluster is down